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ABSTRACT 


This  paper  reviews  the  recommendations  made  by  senior  study  panels,  such  as 
those  of  the  Defense  Science  Board,  the  Army  Science  Board,  and  the  General  Accounting 
Office,  concerning  Defense  modeling  and  simulation.  A  total  of  179  recommendations 
were  identified,  made  by  25  different  study  panels  over  a  period  of  about  16  years.  Overall 
they  provide  a  foundation  for  the  work  of  the  Defense  Modeling  and  Simulation  Office 
(DMSO)  which  was  established  in  1991.  In  this  review,  each  recommendation  was 
categorized  with  regard  to  which  DMSO  General  Objective,  Functional  (Application)  Area, 
and  Technical  Objective  it  addresses.  Each  recommendation  was  also  categorized  with 
regard  to  what  action  (application,  targeted  research  and  development,  technical 
improvement,  modeling  and  simulation  development,  management  improvement,  and 
coordination)  it  advocates.  Common  themes  found  in  the  recommendations  concern 
opportunities  and  needs  for  improved  coordination,  increased  involvement  and 
responsibility  of  higher  management,  more  systematic  and  comprehensive  planning, 
improved  technology,  improved  technical  quality,  increased  information  sharing  and 
feedback,  broadened  perspective,  and  new  and  extended  opportunities  for  modeling  and 
simulation,  including  increased  application  in  system  acquisition,  test,  and  evaluation. 
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SUMMARY 


This  paper  reviews  the  recommendations  for  improving  Defense  modeling  and 
simulation  made  by  senior  study  panels,  such  as  those  of  the  Defense  Science  Board,  the 
Army  Science  Board,  and  the  General  Accounting  Office.  These  recommendations  were 
made  by  25  different  study  panels  over  a  period  of  about  16  years.  Most  of  these  study 
panels  were  sponsored  by  the  Office  of  the  Secretary  of  Defense  (OSD)  or  Army 
organizations. 

A  total  of  179  recommendations  were  identified  for  this  review.  Each 
recommendation: 

•  Was  made  by  a  formally  convened  study  panel  assembled  to  address  a  specific 
matter  of  interest  articulated  in  a  charter  or  terms  of  reference. 

•  Discusses  modeling  or  simulation  explicitly  or  is  included  in  a  study  explicitly 
convened  to  address  Defense  modeling  and/or  simulation  issues. 

•  Is  not  classified. 

•  Is  accessible  through  ordinary  means  (e.g..  Defense  Technical  Information 
Center). 

The  paper  does  not  discuss  why  each  study  panel  made  its  recommendations,  nor 
does  it  comment  on  whether  anything  was  done  to  implement  the  recommendations. 
Nonetheless,  the  fact  that  many  recommendations  were  made  repeatedly  suggests  that  more 
should  be  done  to  implement  them. 

The  recommendations  identified  for  this  review  provide  a  foundation  for  the  work 
of  the  Defense  Modeling  and  Simulation  Office  (DMSO),  which  was  established  in  1991. 
Each  recommendation  was  classified  according  to  the  DMSO  General  Objective,  Functional 
(Application)  Area,  and  Technical  Objective  it  addresses  and  according  to  which  of  six 
actions  it  recommends.  The  results  of  these  classifications  are  tallied  in  Table  S-l. 

Common  themes  that  run  though  the  recommendations  concern  the  opportunities 
and  needs  for:  increased  information  sharing  and  feedback,  increased  involvement  and 
responsibility  of  higher  management,  broadened  perspective,  more  systematic  and 
comprehensive  planning,  improved  technology,  improved  technical  quality,  improved 
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coordination,  and  new  and  extended  applications  for  modeling  and  simulation,  especially 
in  system  acquisition,  test,  and  evaluation. 


Table  S-i.  Tallies  from  Four  Classifications  of  the  Modeling  and 
Simulation  Recommendations 


Recommendations 


General  Objectives 
Management 
Policy 
Investment 

Total 


Functional  Areas 

Education,  Training,  and  Military  Operations 
Research  and  Development 
Test  and  Evaluation 
Analysis 

Production  and  Logistics 

Total 


Technical  Objectives 

Architectural  Development 
Methodology  Development 
Advanced  Technologies 
Information  Sharing 

Total 


Actions 

Application  Area 

Targeted  Research  and  Development 
Technical  Improvement 
Modeling  and  Simulation  Development 
Management  Improvement 
Coordination 

Total 


The  conclusions  drawn  from  this  review  are  the  following: 

•  Interest  in  Defense  modeling  and  simulation  is  widespread  and  persistent 

•  There  are  substantial  needs  and  opportunities  for  sharing  information, 
capabilities,  and  resources  within  and  among  the  Defense  modeling  and 
simulation  communities. 
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•  More  attention  should  be  paid  to  the  verification,  validation,  and  accreditation 
of  Defense  models  and  simulations. 

•  Some  specific  areas,  such  as  the  architectural  issues  of  interoperability  and 
specification  of  standards  and  the  life-cycle  support  of  Defense  models  and 
simulations  themselves,  deserve  more  attention  and  support. 

•  There  are  substantial  needs  and  opportunities  for  improving  the  technologies 
underlying  Defense  modeling  and  simulation. 

•  There  are  many  areas  to  which  Defense  modeling  and  simulation  either  should 
be  applied  anew  or  extended,  especially  those  associated  with  system 
acquisition. 

•  More  needs  to  be  done  to  unite  the  modeling  and  simulator  communities  into  an 
integrated,  coordinated  community  prepared  to  address  the  problems  of 
warfighting. 

•  Joint  warfighting  represents  a  significant  area  for  increased  use  of  modeling 
and  simulation. 

•  There  are  substantial  needs  and  opportunities  for  improving  management  and 
coordination  of  Defense  modeling  and  simulation  activities. 

•  Development  of  standards  is  needed  to  permit  ready  sharing  of  modeling  and 
simulation  data  and  resources. 
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A  REVIEW  OF  STUDY  PANEL  RECOMMENDATIONS  FOR 
DEFENSE  MODELING  AND  SIMULATION 


A.  BACKGROUND 

This  paper  reviews  recommendation s  for  improving  Department  of  Defense  (DoD) 
modeling  and  simulation  made  by  senior  study  panels,  such  as  those  of  the  Defense 
Science  Board  (DSB),  the  Army  Science  Board  (ASB),  and  the  General  Accounting  Office 
(GAO).  The  draft  DoD  directive  on  modeling  and  simulation  gives  the  following 
definitions  for  models  and  simulation: 

•  Model.  A  physical,  mathematical,  or  otherwise  logical  representation  of  a  real- 
world  system,  entity,  phenomenon,  or  process. 

•  Simulation.  A  method  for  implementing  a  model  over  time.  Also,  a  technique 
for  testing,  analysis,  or  training  in  which  real-world  systems  are  used,  or 
where  real-world  and  conceptual  systems  are  reproduced  by  a  model. 

The  definitions  used  by  the  study  panels  for  this  review  were  more  frequently 
implicidy  assumed  than  explicidy  stated,  but  their  recommendations  were  taken  at  face 
value.  If  a  study  panel  or  one  of  its  recommendations  stated  that  it  concerned  modeling 
and/or  simulation,  we  assumed  that  it  did. 

The  study  panels  identified  for  this  review  were  chartered  to  address  specific 
matters  of  interest  to  the  Defense  decision  makers  who  sponsored  them.  Over  the  last  16 
years,  these  panels  have  frequently  noted  both  the  opportunities  and  requirements  for 
Defense  modeling  and  simulation  in  their  recommendations  for  action.  These 
recommendations  provide  a  foundation  for  the  work  of  the  Defense  Modeling  and 
Simulation  Office  (DMSO),  which  was  established  in  1991,  and  the  policies  and 
procedures  it  pursues. 

DMSO  was  established  in  response  to  steady  and  significant  increases  in  the 
technological  opportunity  provided  by  improvements  in  computation,  communication,  and 
graphics  for  modeling  and  simulation  and  in  requirements  for  more  efficient  (1)  research, 
development,  acquisition,  assessment,  and  deployment  of  modem  weapons,  (2)  responses 
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to  continuing  and  rapid  changes  in  the  threat  environment,  and  (3)  responses  to 
environmental  and  budgetary  constraints. 

DMSO  supports  and  promotes  applications  of  modeling  and  simulation  in  the 
Department  of  Defense  to  joint  education  and  training,  research  and  development,  test  and 
evaluation,  and  analysis  by: 

•  Establishing  OSD  cognizance  and  facilitating  coordination  among  DoD 
modeling  and  simulation  activities. 

•  Promoting  the  use  of  interoperability  standards  and  protocols  where 
appropriate. 

•  Stimulating  a  high  return  on  investments  in  joint  use  modeling  and  simulation. 

Based  on  this  charter,  DMSO  has  adopted  the  following  objectives: 

•  Promulgate  standards  to  promote  interoperability  of  the  components  of  the 
modeling  and  simulation  environment 

•  Support  development  of  databases,  tools,  and  methodologies  for  community¬ 
wide  use. 

•  Promote  development  of  a  communications  infrastructure  to  support  integration 
of  joint  modeling  and  simulation  activities. 

•  Facilitate  community-wide  coordination  and  information  sharing  (e.g., 
cataloging  of  existing  models  and  simulations). 

B.  PURPOSE 

This  paper  identifies  study  panel  recommendations  that  have  addressed  Defense 
modeling  and  simulation  and  summarizes  them  in  ways  meant  to  assist  DMSO  and  the 
Defense  modeling  and  simulation  community.  The  paper  does  not  discuss  why  each  study 
panel  made  its  recommendations,  nor  does  it  comment  on  whether  anything  was  done  to 
implement  them.  The  fact  that  many  recommendations  have  been  made  repeatedly  suggests 
that  more  should  be  done  to  implement  them,  but  implementation  is  not  the  issue  addressed 
here.  The  task  was  to  identify,  list,  and  summarize  the  recommendations. 

C.  METHOD 

The  data  for  this  review  were  recommendations  made  by  boards  convened  to  advise 
the  DoD.  Recommendations  come  to  Defense  decision  makers  from  many  different 
sources.  The  criteria  for  including  a  recommendation  in  this  review  were  the  following: 


1 .  It  should  be  included  as  a  specific  recommendation  in  a  formally  convened 
study  panel  (such  as  a  Defense  Science  Board  or  Army  Science  Board  study 
group)  assembled  for  a  specified  period  of  time  to  address  a  specific  matter  of 
interest  articulated  in  a  charter  or  in  terms  of  reference  by  a  sponsor  concerned 
with  the  operations  of  the  DoD.  Recommendations  from  authoritative  groups 
such  as  the  Military  Operations  Research  Society  or  one  of  the  Service 
laboratories,  without  such  a  specific  charter,  were  not  included. 
Recommendations  from  the  United  States  General  Accounting  Office  were 
included. 

2.  It  should  discuss  modeling  and/or  simulation  explicitly  or  be  included  as  a 
recommendation  in  a  study  explicitly  convened  to  address  Defense  modeling 
and/or  simulation.  Additional  recommendations  may  be  relevant  to  Defense 
modeling  and  simulation  but  were  not  included  in  order  to  reduce  ambiguity 
about  which  recommendations  were  and  were  not  candidates  for  inclusion  in 
this  review. 

3.  It  should  not  be  classified.  Few  recommendations  that  would  otherwise  have 
been  included  in  this  review  were  classified,  and  their  inclusion  would  not 
substantially  change  the  nature  of  results  reported  here.  Their  exclusion 
permits  public  release  and  distribution  of  this  document 

4.  It  should  be  accessible  through  ordinary  means.  The  recommendations  that 
were  included  in  this  review  and  the  study  panel  repents  from  which  they  were 
drawn  should  be  readily  accessible  to  members  of  the  Defense  modeling  and 
simulation  community. 

D.  RESULTS 

A  total  of  179  recommendations  from  the  reports  of  25  senior  study  panels  were 
included  in  this  review.  Their  years  of  appearance  and  study  sponsors  are  shown  in 
Table  1.  As  the  table  suggests,  a  variety  of  sponsors  over  the  last  16  years  have  requested 
studies  and  received  recommendations  concerning  Defense  modeling  and  simulation.  The 
table  also  suggests  that  sponsor  interest  in  modeling  and  simulation  may  be  increasing  since 
eight  of  the  25  studies,  with  113  of  the  179  recommendations,  were  sponsored  in  die  last 
4  years.  Most  of  the  sponsors  are  OSD  or  Army  organizations. 

Table  2  shows  the  organizations  that  acted  as  sources  for  organizing  the  study 
panels  and  delivering  their  recommendations  to  sponsors.  In  8  of  the  25  studies  the  study 
organizer  and  its  sponsor  were  the  same.  The  17  remaining  studies  were  organized 
either  by  the  Defense  Science  Board  or  Army  Science  Board,  and  138  of  the  179 
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recommendations  came  from  studies  organized  by  these  two  organizations.  Almost  a  third 
of  the  recommendations  came  from  a  single  1988  Army  Science  Board  study  on  the  use  of 
Army  combat  models  for  analysis  and  training  of  joint  and  combined  operations. 

All  the  recommendations  are  listed  in  reverse  chronological  order  in  Appendix  A 
and  are  presented  nearly  verbatim  from  their  sources.  Many  of  the  study  panel  reports  that 
provide  these  recommendations  are  accessible  through  the  Defense  Technical  Information 
Center  (DTIC),  and  a  DTIC  accession  number  is  given  wherever  possible. 

The  current  organization  of  DMSO  was  used  to  categorize  the  recommendations. 
Currently  DMSO  is  organized  into  three  General  Objectives,  five  Functional  (or 
Application)  Areas,  and  four  Focus  (or  Technical  Objective)  Areas.  All  179  recommen¬ 
dations  were  classified  within  each  of  these  three  areas.  The  General  Objective,  Functional 
Area,  and  Technical  Objective  addressed  by  each  recommendation  are  listed  in  parentheses 
following  its  documentation  in  Appendix  A.  All  179  recommendations  were  also  classified 
according  to  the  actions  they  call  for.  The  particular  action  each  recommendation  calls  for 
is  listed  in  square  brackets  following  its  documentation  in  Appendix  A.  The  tallies  from 
these  classifications  are  shown  in  Table  3. 

As  Table  3  s.ows,  most  of  the  recommendations  concern  the  DMSO  General 
Objectives  of  Management  and  Investment;  most  of  the  recommendations  concern  the 
DMSO  Functional  Areas  of  Education,  Training,  and  Military  Operations  and  Analysis;  and 
most  of  the  recommendations  concern  the  DMSO  Technical  Objective  of  Methodology. 
Most  of  the  actions  called  for  by  the  recommendations  concern  management  improvements. 

1 .  General  Objectives 

All  179  recommendations  were  categorized  according  to  which  single  General 
Objective  they  concern.  These  results  concerning  DMSO  General  Objectives  axe  discussed 
in  more  detail  in  Appendix  B.  The  three  General  Objectives  for  DMSO  axe  the  following; 

a.  Management 

DMSO  should  (1)  establish  a  DoD-wide  management  structure  to  coordinate  joint 
modeling  and  simulation  activities  and  requirements  for  their  use  and  (2)  promote 
coordination  across  DoD  components  to  minimize  duplication  in  modeling  and  simulation 
developments  efforts  and  to  increase  productivity  through  combined  efforts. 
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Table  3.  Tallies  from  Four  Classifications  of  the  Modeling  and 
Simulation  Recommendations 
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General  Objectives 
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Policy 
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Total 
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Information  Sharing 
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Actions 

Application  Area 

Targeted  Research  and  Development 
Technical  Improvement 
Modeling  and  Simulation  Development 
Management  Improvement 
Coordination 

_  Total 


Ninety  (SO  percent)  of  the  179  recommendations  concern  the  General  Objective  of 
Management  The  Management  recommendations  by  themselves  are  about  evenly  split 
between  the  Functional  Areas  of  Education,  Training,  and  Military  Operations  on  one  hand 
(34  percent)  and  Analysis  (37  percent)  on  the  other.  Most  of  the  Management 
recommendations  concern  the  Technical  Objective  of  Methodology  (67  percent). 
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b.  Policy 

DMSO  should  (1)  promulgate  and  oversee  the  implementation  of  a  DoD  modeling 
and  simulation  policy;  (2)  develop  policy  in  specific  areas  necessary  to  ensure  the  effective 
joint  application  of  models  and  simulations;  (3)  fix  responsibilities  to  ensure  the  proper 
oversight  of  models  and  simulations  with  joint  applicability;  and  (4)  furnish  guidance  for 
the  consistent  development  of  modeling  and  simulation  plans  by  the  DoD  components. 

Fourteen  (8  percent)  of  the  179  recommendations  concern  the  General  Objective  of 
Policy.  Most  of  the  Policy  recommendations  concern  the  Test  and  Evaluation  Functional 
Area  (50  percent)  and  the  Technical  Objective  of  Methodology  (71  percent). 

c.  Investment 

DMSO  should  (1)  formulate  and  implement  a  long-range  joint  investment  strategy 
for  models  and  simulations  and  (2)  identify  and  fund  high-priority  investments  leading  to 
enhanced  joint  modeling  and  simulation  capabilities. 

Seventy-five  (42  percent)  of  the  179  recommendations  concern  the  General 
Objective  of  Investment.  These  recommendations  are  about  evenly  split  between  the 
Functional  Areas  of  Education,  Training,  and  Military  Operations  (40  percent)  on  one  hand 
and  Analysis  (37  percent)  on  the  other.  Most  of  the  Policy  recommendations  concern  the 
Technical  Objective  of  Methodology. 

2.  Functional  (Application)  Areas 

All  179  recommendations  were  categorized  according  to  which  single  Functional 
Area  in  DMSO  they  concern.  The  results  concerning  DMSO  Functional  (Application) 
Areas  are  discussed  in  more  detail  in  Appendix  C.  The  5  Functional  Areas  are  the 
following: 


a.  Education,  Training,  and  Military  Operations  (ET&MO) 

These  applications  emphasize  modeling  and  simulation  applications  for  training  in 
joint  military  operations.  They  also  include  modeling  and  simulation  applications  such  as 
the  re-creation  of  historical  battles,  doctrine  and  tactics  development,  command  and  unit 
training,  operational  planning  and  rehearsal,  and  wartime  situation  assessments. 

Sixty-three  (35  percent)  of  the  179  recommendations  concern  modeling  and 
simulation  applications  in  Education,  Training,  and  Military  Operations.  These 
recommendations  are  about  evenly  split  between  the  General  Objectives  of  Management 
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(49  percent)  and  Investment  (48  percent).  Most  of  them  concern  the  Technical  Objective  of 
Methodology  (81  percent). 

b.  Research  and  Development  (R&D) 

These  applications  include  modeling  and  simulation  applications  such  as  the 
definition  of  requirements,  support  for  engineering  design,  and  assessments  of  system 
performance.  They  focus  on  articulating  the  requirements  of  the  research  and  development 
community  and  managing  its  activities.  They  concern  modeling  ami  simulation  applications 
that  are  used  for  exploration  by  scientists  and  engineers  and  that  support  the  needs  of  the 
research  and  development  community. 

Twenty-three  (13  percent)  of  the  179  recommendations  concern  modeling  and 
simulation  applications  in  Research  and  Development.  Most  of  these  recommendations 
concern  the  General  Objective  of  Management  (37  percent)  and  the  Technical  Objective  of 
Methodology  (52  percent). 

c.  Test  and  Evaluation  (T&E) 

These  applications  include  modeling  and  simulation  applications  such  as  early 
operational  assessments,  design  of  operational  tests,  and  excursion  and  sensitivity 
analyses.  They  focus  on  modeling  and  simulation  applications  that  enhance  the  quality  of 
test  and  evaluation  end  products  by  overcoming  test  impediments  and  test  resource 
limitations. 

Twenty-four  (13  percent)  of  the  179  recommendations  concern  modeling  and 
simulation  applications  in  Test  and  Evaluation.  Most  of  these  recommendations  concern 
the  General  Objective  of  Management  (46  percent)  and  the  Technical  Objective  of 
Methodology  (75  percent). 

d.  Analysis 

These  applications  include  modeling  and  simulation  applications  such  as  campaign 
analyses,  assessing  force  structure,  determining  system  configuration  determination,  and 
cost  analyses.  They  focus  on  modeling  and  simulation  applications  that  inform  decisions 
concerning  the  development  of  Defense  operational  capabilities. 

Sixty-five  (36  percent)  of  the  179  recommendations  concern  modeling  and 
simulation  applications  in  Analysis.  These  recommendations  are  evenly  split  between  the 
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General  Objectives  of  Management  (51  percent)  and  Investment  (43  percent)  and  emphasize 
the  Technical  Objective  of  Methodology  (60  percent). 

e.  Production  and  Logistics  (P&L) 

These  applications  include  modeling  and  simulation  applications  such  as  system 
producibility  assessments,  industrial  base  appraisals,  and  determinations  of  logistics 
requirements.  They  focus  on  modeling  and  simulation  applications  that  support  procuring, 
maintaining,  and  transporting  Defense  materiel,  personnel,  and  facilities. 

Only  four  (2  percent)  of  the  179  recommendations  concern  modeling  and  simulation 
applications  in  Production  and  Logistics.  Two  address  the  General  Objective  of 
Management  and  two  concern  Investment;  all  four  address  the  Technical  Objective  of 
Methodology. 

3 .  Technical  Objectives 

All  179  recommendations  were  categorized  according  to  which  single  Technical 
Objective  they  concern.  These  results  concerning  DMSO  Technical  Objectives  are 
discussed  in  more  detail  in  Appendix  D.  The  four  Technical  Objectives  are  the  following: 

a.  Architectural  Development 

This  objective  is  to  develop  and  promulgate  standards  that  promote  the 
interoperability  of  modeling  and  simulation  components.  It  concerns  architectural 
structures  that  allow  interoperability  and  sharing  of  assets  and  specification  of  interface 
standards  to  support  the  modeling  and  simulation  user  communities. 

Only  one  recommendation  concerns  Architectural  Development  It  addresses  the 
General  Objective  of  Investment  and  the  Functional  Area  of  Education,  Training,  and 
Military  Operations. 

b.  Methodological  Development 

This  objective  is  to  develop  and  promulgate  databases,  tools,  and  methodologies  for 
community-wide  use.  It  focuses  on  the  conceptual  bases  and  procedures  for  the 
verification,  validation,  and  accreditation  of  models  and  simulations. 

One  hundred  twenty-three  (69  percent)  of  the  179  recommendations  concern 
Methodology.  The  closeness  with  which  these  recommendations  are  linked  to 
methodological  considerations  in  modeling  and  simulation  varies  widely.  These 
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recommendations  are  evenly  split  between  the  General  Objectives  of  Management 
(60  percent)  and  Investment  (52  percent)  and  between  the  Functional  Areas  of  Education, 
Training,  and  Military  Operations  (51  percent)  on  one  hand  and  Analysis  (38  percent)  on 
the  other. 

c.  Advanced  Technologies 

This  objective  is  to  develop  and  promulgate  key  technologies  for  supporting 
modeling  and  simulation.  These  technologies  include  computer  hardware,  software 
engineering,  graphics  displays,  and  behavioral  representation. 

Advanced  Technology  could  reasonably  be  combined  with  Methodology  as  a 
Technical  Objective,  but  it  is  kept  separate  for  this  analysis.  Eighteen  (10  percent)  of  the 
179  recommendations  concern  Advanced  Technology.  These  recommendations  emphasize 
the  General  Objective  of  Investment  (78  percent)  and  the  Functional  Area  of  Research  and 
Development  (50  percent). 

d.  Information  Sharing 

This  objective  is  to  develop  and  promote  coordination  and  communication  across 
and  within  the  Defense  modeling  and  simulation  communities. 

Information  Sharing  could  reasonably  be  combined  with  Methodology  as  a 
Technical  Objective,  but  it  is  kept  separate  for  this  analysis.  Thirty-seven  (21  percent)  of 
the  179  recommendations  concern  Information  Sharing.  These  recommendations 
emphasize  the  General  Objective  of  Management  (73  percent)  and  the  Functional  Area  of 
Analysis  (62  percent). 

4 .  Recommended  Actions 

All  179  recommendations  were  categorized  according  to  the  actions  they  call  for. 
These  results  concerning  Recommended  Actions  are  discussed  in  more  detail  in 
Appendix  E.  The  six  recommended  actions  are  the  following: 

a.  Application  Area 

Recommendations  for  applications  of  modeling  and  simulation  to  new  areas  of 
activity  or  expansions  of  existing  applications  where  the  emphasis  was  on  the  area  of 
application  and  not  on  the  development  of  new  or  expanded  modeling  and  simulation 
capabilities  were  classified  as  application  recommendations. 
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Many  study  panels,  addressing  various  issues  of  military  policy  and  practice,  have 
recommended  modeling  and  simulation  as  means  to  resolve  these  issues.  Twenty-nine 
(16  percent)  of  the  179  recommendations  concern  either  applications  of  modeling  and 
simulation  to  new  areas  of  activity  or  expansions  of  the  use  of  modeling  and  simulation  in 
already  existing  areas  of  application. 

b.  Targeted  Research  and  Development 

Recommendations  for  research  and  development  of  new  or  expanded  capabilities 
for  modeling  and  simulation  where  the  emphasis  was  on  the  new  or  expanded  capabilities 
and  not  on  new  or  expanded  areas  of  applications  for  modeling  and  simulation  were 
classified  as  research  and  development  recommendations. 

Many  study  panels  have  noted  requirements  for  increased  capabilities  in  the 
technology  of  modeling  and  simulation.  Thirty-four  (19  percent)  of  the  179  recommen¬ 
dations  concern  research  and  development  of  new  or  expanded  capabilities.  About  two- 
thirds  of  these  recommendations  concern  specific  capabilities  that  should  be  developed,  and 
about  one-third  concern  the  management  of  programs  to  accomplish  these  ends. 

c.  Technical  Improvement 

Recommendations  for  improvements  in  the  conduct  of  modeling  and  simulation 
activities  that  primarily  require  better  use  of  existing  technical  capabilities  rather  than  the 
development  of  new  ones  were  classified  as  recommendations  for  technical  improvements. 

Many  study  panels  have  noted  requirements  to  extend  application  of  existing 
technical  capabilities  in  the  conduct  of  modeling  and  simulation.  Thirty-nine  (22  percent) 
of  the  179  recommendations  concern  improvements  that  require  better  use  of  existing 
technical  capabilities. 

d.  Modeling  and  Simulation  Development 

Recommendations  for  improvements  in  the  design  and  development  of  models  and 
simulations  were  classified  as  recommendations  for  modeling  and  simulation  preparation. 

Twelve  (7  percent)  of  the  179  recommendations  concern  improvements  in  the 
design  and  development  of  models  and  simulations.  Almost  all  of  these  recommendations 
concern  the  development  of  models  and  simulations  used  in  the  acquisition,  test,  and 
evaluation  of  other  systems. 
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e.  Management  Improvement 

Recommendations  for  improvements  in  the  conduct,  control,  and  use  of  modeling 
and  simulation  were  classified  as  recommendations  for  management  improvements. 

Forty-five  (25  percent)  of  the  179  recommendations  concern  improvements  in  the 
way  modeling  and  simulation  activities  are  conducted,  controlled,  and  used. 

f.  Coordination 

Recommendations  for  coordinating  under  single  management  or  in  a  single  location 
modeling  and  simulation  activities  or  resources  such  as  algorithms  and  databases  were 
classified  as  recommendations  for  coordination  and  centralization. 

Twenty  (11  percent)  of  the  179  recommendations  concern  the  coordination  of 
modeling  and  simulation  activities  or  resources  under  under  single  management  or  in  a 
single  location.  These  coordination  recommendations  are  a  subset  of  the  management 
improvements  discussed  above. 

E.  COMMON  THEMES 

Some  common  themes  run  through  these  179  recommendations  and  across  most  of 
the  16  years  in  which  they  were  made.  These  themes  include: 

•  Increased  information  sharing  and  feedback.  Defense  management  should 
provide  greater  cooperation  and  information  sharing  on  the  development, 
implementation,  maintenance,  verification,  validation,  and  accreditation  of 
Defense  models  and  simulations  (1)  within  each  of  the  Services,  (2)  among  the 
Services,  the  Joint  Staff,  and  the  CINCs,  and  (3)  between  the  U.S.  Defense 
establishment  and  its  allies. 

•  Higher  level  management  responsibility.  Management  responsibility  for 
models  and  simulations  should  be  raised  to  senior  levels  of  the  Services  and 
Joint  Staff.  Without  this  level  of  attention  and  responsibility,  different  and 
perhaps  inconsistent  assumptions  and  methods  will  continue  to  be  used  to 
make  critical  decisions  concerning  Defense  policy  and  procedures. 
Responsibility  for  quality,  control,  and  dissemination  of  models  and 
simulations  and  of  their  data,  algorithms,  and  resources  should  be  more 
coordinated  and  centralized. 

•  Broadened  perspective.  The  general  perspective  on  how  and  where  modeling 
and  simulation  technology  should  be  applied  should  be  extended  to  new  areas 
and  broadened  within  existing  areas  of  application.  Military  science  should  be 
fully  integrated  in  the  development  and  use  of  Defense  models  and  simulations. 


•  Systematic  planning.  The  requirements,  resources,  and  responsibilities  for 
Defense  modeling  and  simulation  should  be  systematically  developed, 
documented,  and  reviewed,  periodically  revised,  and  then  used  to  guide 
management  of  Defense  modeling  and  simulation.  Deficiencies  in  Defense 
modeling  and  simulation  should  be  identified  and  prioritized,  applicable 
technology  should  be  forecast,  and  this  information  should  be  systematically 
applied  to  guide  investments  in  Defense  modeling  and  simulation. 

•  Improved  technology.  Significant  opportunities  exist  for  improving  the 
technology  of  both  modeling  and  simulation.  Specific  investments  can  and 
should  be  made  to  improve  the  basic  technology  of  modeling  and  simulation, 
for  instance  by  developing  and  integrating  accurate  factors  for  human 
performance  in  models  and  simulations,  devising  computer  tools  for 
developing  and  testing  models  and  simulations,  developing  intelligent 
computer  based  adversaries,  developing  joint  threat  simulators,  reducing 
controller  requirements,  and  increasing  simulation  portability. 

•  Improved  technical  quality.  Significant  opportunities  exist  for  improving  the 
technical  quality  of  Defense  modeling  and  simulation.  Specific  investments 
can  and  should  be  made  to  raise  the  technical  quality  of  models  and 
simulations,  for  instance,  by  incorporating  instructional  technology  into  flight 
simulators,  improving  the  statistics,  distributions  and  assessment  routines  used 
in  many  models,  ensuring  assignment  of  qualified  personnel  to  modeling  and 
simulation  positions,  exploiting  commercial  developments  such  as  new 
graphics  capabilities,  distributed  computing,  and  networking,  allowing  remote 
participation  by  non-collocated  participants  including  U.S.  allies,  requiring 
third  party  and/or  senior  review  of  models  and  simulations,  and  increasing  the 
interoperability  of  Defense  models  and  simulations. 

•  Increased  application  in  system  acquisition,  test,  and  evaluation.  This  single 
area  of  application  is  mentioned  so  often  across  the  recommendations  that  it 
deserves  separate  presentation.  Modeling  and  simulation  should  be  used  early 
and  repeatedly  in  system  acquisition  to  improve  our  understanding  of  how  new 
systems  will  perform  in  combat  before  they  are  committed  to  prototyping. 

•  New  and  extended  applications.  Specific  investments  can  and  should  be  made 
to  extend  modeling  and  simulation  to  new  areas  and  within  existing  areas  of 
application  such  as  crew,  group,  team,  and  unit  training,  military  operations  in 
built  up  areas,  combat  service  support  planning,  ground  attack  munitions, 
echelon  above  corps,1  and  joint  warfare. 


1  Discussions  of  echelon  above  corps  (EAC)  modeling  usually  focus  on  the  need  for  joint  simulation.  It 
should  be  noted  that  joint  operations  can  be  performed  at  any  level,  not  just  at  echelons  above  corps, 
that  recommendations  of  this  sort  may  be  motivated  as  much  by  a  desire  for  joint  simulation  as  for 
high-level  simulation,  and  that  these  recommendations  may  be  looking  for  two  fairly  independent 
capabilities,  not  just  one. 


F.  CONCLUSIONS 


This  review  has  been  primarily  descriptive,  but  it  suggests  the  following 
conclusions: 

•  Interest  in  Defense  modeling  and  simulation  is  widespread  and  persistent. 
Many  study  panels  have  recommended  the  application  of  modeling  and 
simulation  to  many  areas.  This  widespread  interest  is  evident  across  all  16 
years  of  study  panels  summarized  here.  Additionally,  many  study  panels  have 
been  given  as  their  primary  topic  some  aspect  of  Defense  modeling  and 
simulation. 

•  There  are  substantial  needs  and  opportunities  for  sharing  information, 
capabilities,  and  resources  within  and  among  the  Defense  modeling  and 
simulation  communities.  This  conclusion  is  based  on  the  frequency  with 
which  recommendations  for  sharing  of  information,  capabilities,  and  resources 
are  made  by  senior  study  panels. 

•  More  attention  should  be  paid  to  the  verification,  validation,  and  accreditation 
of  Defense  models  and  simulations.  Many  recommendations  addressed  the 
desirability  of  increased  explicit  and  impartial  verification  and  validation  of 
Defense  models  and  simulations. 

•  Some  specific  areas  of  Defense  modeling  and  simulation  development  and 
implementation  deserve  more  attention,  such  as  the  architectural  issues  of 
interoperability  and  specification  of  interface  standards  for  modeling  and 
simulation.  Given  the  current  interest  in  networiting  simulations  and  in  linking 
different  models  and  simulation  together,  attention  to  architectural  issues  seems 
likely  to  grow.  There  has  also  been  little  attention  given  to  systematic  life-cycle 
support  of  models  and  simulations  themselves  and  more  is  needed. 

•  There  are  substantial  needs  and  opportunities  for  improving  the  technologies 
underlying  Defense  modeling  and  simulation.  The  frequency  with  which 
technical  improvements  are  recommended  by  senior  study  panels  strongly 
suggests  that  there  are  many  technical  opportunities  and  improvements  to  be 
pursued  and  that  these  opportunities  have  been  overlooked  by  the  Defense 
modeling  and  simulation  communities. 

•  There  are  many  areas  to  which  modeling  and  simulation  either  should  be 
applied  anew  or  extended.  Most  prominent  among  these  areas  are  those 
associated  with  acquisition-"simulate  before  you  buy"  and  applying  simulation 
through  every  stage  of  the  acquisition  process.  However,  the  wide  range  of 
other  application  areas  recommended  suggests  that  there  are  at  least  some,  if 
not  many,  new  areas  to  which  modeling  and  simulation  should  be  applied  such 
as  logistics  and  combat  engineering. 
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•  Many  study  panels  have  focused  almost  exclusively  on  either  simulator 
technology  or  computerized  wargaming.  Substantial  work  remains  to  be  done 
to  build  consensus  and  collegiality  and  to  unite  people  concerned  with  either 
modeling  or  simulation  into  a  coordinated  community  prepared  to  address  the 
problems  of  warfighting. 

•  Joint  applications  of  modeling  and  simulation  receive  emphasis  chronologically 
late  in  the  stream  of  recommendations  considered  here,  but  this  later  emphasis 
is  substantial  and  frequent.  Joint  warfighting  represents  a  significant  area  for 
increased  use  of  modeling  and  simulation  technology. 

•  There  are  substantial  needs  and  opportunities  fen*  improving  management  and 
coordinated  development  of  Defense  modeling  and  simulation  activities.  The 
fact  that  such  recommendations  have  been  made  repeatedly  suggests  by  itself 
the  need  for  attention  from  senior  management. 

•  Together,  the  recommendations  for  coordination  and  information  sharing 
suggest  a  need  to  develop  standards  that  permit  ready  sharing  of  data  and 
resources  among  models,  simulations,  and  simulators  and  to  allow  them  to  be 
networked. 

Finally,  it  should  be  noted  that  most  of  the  recommendations  reviewed  here  came 
from  study  panels  sponsored  either  by  OSD  organizations  or  the  Army  and  that  the  above 
themes  and  conclusions  are  based  primarily  on  recommendations  made  to  these  sponsors. 
Recommendations  from  other  sponsors,  especially  those  in  the  Navy  and  Air  Force,  were 
not  found  through  the  standard  channels  used  to  locate  information  for  this  review. 


APPENDIX  A 

MODELING  AND  SIMULATION  RECOMMENDATIONS 


1 .  [ASB,  1991]  ARMY  SCIENCE  BOARD  1991  SUMMER  STUDY  ON 
ARMY  SIMULATION  STRATEGY  (AUGUST  1991).  WASHINGTON, 
DC:  ARMY  SCIENCE  BOARD.  (DRAFT)  [SPONSORED  BY  THE 
DEPUTY  UNDER  SECRETARY  OF  THE  ARMY  FOR  OPERATIONS 
RESEARCH] 


Overall 

•  The  Secretary  of  the  Army  and  the  Chief  of  Staff  should  assure  that  the  Army 
proceeds  into  the  Electronic  Battlefield  era  as  rapidly  and  enthusiastically  as 
possible. 

(Policy,  ET&MQ.  Methodology)  [Technical  Improvement! 

•  The  Electronic  Battlefield  program  should  be  funded  at  the  Department  of  the 
Army  level.  It  should  have  a  single  manager  with  a  direct  mandate  from  the 
Secretary  of  the  Army. 

(Management.  ET&MQ.  Methodology)  [Coordinate! 

•  In  the  future  the  development  of  major  materiel  systems  should  normally 
involve  experimentation  and  testing  using  electronic  prototypes  at  every  stage 
of  development  from  requirements  generation  to  testing  of  production  articles. 
Testing  of  electronic  systems  on  the  Electronic  Battlefield  will  not  obviate  the 
need  for  testing  of  actual  articles  in  the  real  environment  The  two  should 
become  parts  of  a  complementary  process  that  is  far  superior  to  that  which  has 
been  done  before. 

(Management  T&E.  Methodology)  [Application! 

Training  on  the  Electronic  Battlefield 

•  The  Army  should  review  the  current  structure  of  the  Electronic  Battlefield 
program. 

(Management,  ET&MQ,  Methodology)  [Management  Improvement! 
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•  The  Army  should  enthusiastically  adopt  electronic  training  as  a  major  part  of  its 
training  strategy. 

(PqHcv  .  et&mo.  Mcthodatogy)  \  Application! 

•  The  Army  should  be  willing  to  trade-off  OPTEMPO  as  well  as  other  funds  to 
finance  die  new  modes  of  training. 

(Management.  ET&MO.  Methodology)  Management  Improvement! 

•  The  Army  should  adopt  a  combined  arms  simulation  strategy  from  the  outset 
and  should  look  for  a  scheme  that  will  provide  at  least  a  combined  arms 
capability  at  the  Battalion  Command  Held  Exercise  level  at  die  earliest  possible 
moment 


(Investment,  ET&MQ,  Methodology) 


[Technical  Ii 


it] 


•  The  Army  should  fund  these  programs  at  the  Department  of  the  Army  level  and 
should  undertake  cost  effective  actions  that  build  on  existing  SIMNET 
hardware  and  software. 


(Management.  ET&MO.  Methodology)  fTargeted  R&D1 


Development  and  Testing  on  the  Electronic  Battlefield 

•  The  Army  should  mandate  the  use  and  sharing  of  electronic  prototyping 
through  all  stages  of  system  development.  Refined  versions  should  be 
provided  back  to  the  combat  developers  for  their  continued  use  and 
participation  with  the  materiel  development  community,  as  system  definition 
proceeds.  Physical  testing  should  only  be  used  for  limited  confirmation  tests 
in  key  areas  and  for  testing  (such  as  reliability)  that  cannot  reasonably  be 
conducted  with  the  electronic  prototype. 

(Policy,  I&E,  Methodology)  f  Application! 


Managing  the  Electronic  Battlefield 

•  A  single  manager  should  be  put  in  charge  of  the  Electronic  Battlefield--its 
design,  fielding,  and  operation-as  a  service  system  to  be  used  by  the 
numerous  Army  customers.  Two  existing  programs,  CATT  and  BDS-D, 
should  be  combined  by  making  them  extensions  of  the  Electronic  Battlefield. 
The  manager  should  also  resolve  issues  of  whether  and  how  to  upgrade 
SIMNET  or  to  retire  it  as  the  current  plan  envisages.  The  Army  should  use 
this  manager  as  a  key  point  of  contact  on  distributed  simulation  with  similar 
managers  in  other  services  and  with  the  new  Defense  Modeling  and  Simulation 
Office.  This  manager  should  be  a  flag  officer  PEO  whose  office  is  a  small, 
lean  management  cell  supported  by  other  parts  of  the  Army  and  assisted  in  the 
day-to-day  operation  of  the  system  by  contractors  or  possibly  an  FFRDC. 


(Management,  ET&MO.  Methodology)  [Coordinate! 


Data  Base  and  Model  Quality 

•  The  Army  should  develop  a  process  and  discipline  to  ensure  quality  of  data 
across  the  total  spectrum  of  model  use.  This  should  be  a  prime  task  for  the 
manager  of  the  Electronic  Battlefield.  A  limited  number  of  databases  and 
models  will  be  widely  used  in  the  Electronic  Battlefield.  The  Electronic 
Battlefield  manager  should  assure  that  these  are  "certified"  to  ensure  validity 
and  consistency.  Other  databases  and  models  may  be  made  available  for  use 
centrally  from  the  Electronic  Battlefield  even  though  they  are  not  officially 
"certified"  so  that  new  concepts  can  be  introduced  for  examination  and  testing 
before  they  are  validated. 

(Management  ET&MO.  Methodology!  [Coordinate] 

Technology  Investment  Strategy 

•  Because  of  the  existing  strong  commercial  impetus  for  technology 
development,  the  Army  should  utilize  technology  advances  made  in  the  outside 
world.  It  should  adopt  a  strategy  of  exploiting  technology  that  is  developed 
elsewhere  and  should  not  invest  in  technology  development  for  modeling  and 
simulation  except  for  specific  Army-unique  needs. 

(Policy.  R&D-  Advanced  Technology)  fM&S  Development! 

2.  [FM&P,  1990]  SIMULATION  POLICY  STUDY  (1990). 

WASHINGTON,  DC:  OFFICE  OF  THE  ASSISTANT  SECRETARY 
OF  DEFENSE  (FORCE  MANAGEMENT  AND  PERSONNEL)  AND 
OFFICE  OF  THE  DIRECTOR  OF  DEFENSE  RESEARCH  AND 
ENGINEERING. 

•  Set  as  your  objective  interoperable,  distributed  M&S  capable  of  depicting 
STRATEGIC  FORCE  PROJECTION,  consistent  with  the  Defense  Policy 
Guidance,  whether  for  regional  contingencies,  or  for  reconstituting  theater 
defenses  in  Europe  or  the  Northwest  Pacific.  Invest  to  support  the  CINCs  of 
the  combatant  commands  of  the  United  States,  for  since  U.S.  force  will  fight 
only  under  their  command,  M&S  should  reflect  that  reality.  Invest  to 
reconfigure  M&S  to  adapt  rapidly  to  new  threats  and  unanticipated 
contingencies.  Invest  to  treat  in  M&S  the  Reserve  Components  as  mainline 
elements  of  the  Total  Force. 

(Investment.  Analysis.  Methodology)  [Targeted  R&D1 
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Approve  and  implement  the  EXCIMS/DMSO  management  scheme.  Assign 
DARPA  immediate  action,  pending  approval  and  staffing,  and  plan  for 
DARPA's  passing  overall  management  to  the  DMSO  not  later  than  start  of 
FY  1992,  while  remaining  the  lead  OSD  actor  in  R&D.  We  estimate  this  will 
require  5  to  10  percent  of  available  funds. 

(Management.  R&D.  Methodology)  (Management  Improvement! 

Invest  15  percent  of  your  available  M&S  funds  in  basic  research  to  bring  to 
DoD’s  M&S  the  best  mathematical  paradigms,  analytical  methods,  and 
interoperability  techniques  available  in  America's  universities,  laboratories,  and 
industry. 

(Investment.  R&D.  Methodology!  (Targeted  R&D1 

Invest  15  percent  of  your  M&S  funds  in  enabling  technologies:  more 
powerful,  cheaper  processors,  graphic  displays,  data  base  construction,  and 
communications. 


(Investment.  R&D.  Advanced  Technology) 


(Technical  Ii 


Invest  60  percent  of  your  available  M&S  funds  to  develop  a  prototype, 
transportable,  hybrid  representation  of  an  operational  theater  that  draws  on 
empirically-derived  representation  of  the  behavior  of  actual  physical  and  human 
systems.  Observations  of  that  behavior  should  be  sought  in  actual  operations, 
in  operational  tests,  or  in  training  exercises  that  most  closely  replicate  the 
complexities  and  frictions  of  battle,  such  as  large-scale  Tactical  Engagement 
Simulation  (TES).  Use  the  most  effective  mathematics  to  model  such 
behaviors,  incorporate  actual  equipment  whenever  it  is  cost  effective  to  do  so 
and  aim  at  consistency  across  models,  simulations,  and  operations:  per  the 
present  term  of  art,  seamless  simulation. 


(Investment,  ET&MO.  Methodology)  (Targeted  r&di 

Task  the  Director,  DARPA,  to  undertake  immediately  the  investments  outlined 
above,  drawing  on  the  funds  now  available,  and  securing  cooperation  with  the 
components  to  insure  meshing  new  starts  with  existing  and  near-term  systems, 
smooth  fielding  of  prototypes,  and  managing  new  products  through  their  life- 
cycle. 


(Management.  R&D.  Methodology^  (Targeted  R&DI 

Establish,  in  concert  with  the  Joint  Chiefs  of  Staff,  policies  to  assure  full 
integration  of  military  science  with  the  acquisition  of  future  M&S,  including 
provision  for  educating  military  service  personnel  in  M&S  developments  and 
applications. 


*1 


(Policy.  Analysis.  Methodology) 


(Management  Ii 


3 .  [DSB,  1989]  REPORT  OF  THE  DEFENSE  SCIENCE  BOARD  TASK 
FORCE  ON  IMPROVING  TEST  AND  EVALUATION 
EFFECTIVENESS  (DECEMBER  1989).  WASHINGTON,  DC: 
DEFENSE  SCIENCE  BOARD.  [SPONSORED  BY  THE  UNDER 
SECRETARY  OF  DEFENSE  FOR  ACQUISITION) 


For  the  role  of  models  and  simulations  in  the  evaluation  process: 

•  Hie  Service  acquisition  executives  should  ensure  that  M/S  excursion  analyses 
are  applied  systemically  to  help  reach  and  maintain  agreement  on  major  aspects 
of  requirements  and  system  performance. 

(Management  T&E.  Methodology)  [Application! 

•  The  DOT&E  and  Service  OT&E  communities  should  be  chartered  to  participate 
early  in  the  requirements  process.  In  particular,  they  should  translate 
operational  requirements  into  an  evaluation  framework  and  document  the  roles 
for  M/S  as  well  as  for  testing  in  meeting  evaluation  objectives  at  each  milestone 
and  as  appropriate  between  milestones. 

(Policy.  T&E.  Methodology)  [Application! 

•  The  USD(A)  should  establish  policy  and  provide  guidance  to  the  acquisition 
community  for  systematically  re-evaluating  system  specifications  using  M/S 
and  test  results. 


(Policy.  T&E.  Methodology)  [Application! 

•  Service  acquisition  executives  should  ensure  that  the  development  programs 
employ  man-in-the-loop  simulation  beginning  with  requirements  definition  and 
mature  the  simulation  along  with  the  hardware  throughout  the  acquisition 
process. 


(Management.  T&E.  Methodology!  [Application! 

JCS/CINCs  should  exploit  technology  capabilities  in  distributed  computing 
and  networking  to  simulate  coordinated  combined  arms  engagements  with 
man-in-the-loop  simulations  and  to  evaluate  results  against  live  exercises. 


(Investment.  T&E.  Advanced  Technology! 


(Technical  Ii 


u 


For  the  credibility  of  models  and  simulations: 

•  USD(A)  should  ensure  refinement,  maintenance,  and  availability  of  models, 
weapon  and  threat  data  descriptions,  and  simulation  elements  having  wide  DoD 
utility. 

~  Appropriate  JCS  and  OSD  offices  should  select/fund  executive  agents  to 
maintain  element  repositories  (DNA-Nuclear  models,  DIA-Threat  data. 
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JTCGs,  etc.)  complete  with  databases,  code  libraries,  and  documented 
limitations. 

(Policy.  T&E.  Information  Sharing  [Coordinate! 

•  USD(A)  should  charter  DDR&E  to  enable,  as  necessary,  independent  panels  of 
experts  to  assess  specific  applications  of  M/S  results  on  which  acquisition 
decisions  are  based.  The  work  would  be  tasked  on  a  case-by-case  basis  and 
include  participants  from  academia,  industry,  and  the  government 

(Management  I&E.  Methodology)  [Management  Improvement! 

•  USD(A)  should  modify  DoDI  5000.2  to  require  that  DAB  documentation 
(SCP/DCP,  TEMP,  COEA  and  CAIG)  address  the  applicability  of  models  and 
simulation.  For  example,  the  documentation  could  consider  the  M/S  plan  and 
methodology,  limitations,  assumptions,  extrapolations,  sensitivities,  results, 
analysis,  and  validation. 

(Efllisy,  I&E,  Methodology)  [Management.ImcrovementI 

•  DDR&E  should  continue  to  fund  M/S  technology  at  both  the  fundamental  and 
application  levels,  including  the  M/S  interfaces  and  languages,  executable 
specifications,  model  interoperability,  validation  techniques  and  tools,  and 
parallel  and  networked  simulations. 

(laYfistment,  MD.  Methodology)  [Targeted  R&D1 

[ASB,  1989]  FINAL  REPORT  OF  THE  AD  HOC  SUBGROUP  ON 
CLOSE  COMBAT  (HEAVY)  TRAINING  STRATEGY  FOR  THE  1990'S 
(MARCH  1989).  WASHINGTON,  DC:  ARMY  SCIENCE  BOARD. 
[SPONSORED  BY  THE  ASSISTANT  SECRETARY  OF  THE  ARMY 
FOR  RESEARCH,  DEVELOPMENT,  AND  ACQUISITION] 
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•  Training  for  different  echelons  needs  to  be  reconciled  and  stabilized.  DCSOPS 
and  MACOMS  should  place  increased  emphasis  on  command  and  staff 
simulations  and  training  without  troops  to  allow  higher  echelons  to  train  up 
prior  to  field  exercises. 

(Management,  ET&MQ,  Methodology)  [Technical  Improvement! 

•  The  philosophy  "train  as  you  fight"  requires  access  to  realistic  battlefield 
conditions  to  achieve  the  motivation  and  discipline  to  perform  well  under 
stressful  combat  conditions.  New  technology  additions  and  insertions 
properly  analyzed,  integrated,  and  applied  selectively  have  the  potential  for 
improvement  in  heavy  force  readiness,  particularly  for  the  Reserve 


Components  (RC).  Use  of  networked  simulation  that  is  applicable  to  both  the 
Active  Component  (AC)  and  RC  should  be  expanded. 

(Management.  ET&MO.  Advanced  Technology)  [Targeted  R&D1 

•  Combat  training  for  the  RC  needs  further  alteration  and  tailoring  to  fit  the 
unique  RC  environments.  DCSOPS  should  re-examine  the  applicability  of  AC 
training  devices  to  RC  needs  and  circumstances,  and  increase  the  level  of  RC 
combat  training,  based  on  prioritized  needs  of  specific  RC  units,  through 
increased  and  integrated  use  of  simulation,  networking,  and  specific  Re¬ 
oriented  training  devices. 

(Management.  ET&MO.  Methodology)  [Technical  Improvement) 

•  The  effectiveness  of  various  combinations  of  training  (live-fire,  maneuver, 
simulation)  is  not  being  determined  objectively.  TRADOC  should  establish 
field  and  simulator  test-beds  and  supporting  simulation  models  for  new 
training  strategies,  devices,  and  techniques  comparable  to  or  coordinated  with 
materiel  testing  facilities. 


(Management  ET&MO.  Methodology)  [Technical, I 


ifl 


Manprint  does  not  influence  training  adequately  during  materiel  acquisition. 
DCSPER,  TRADOC,  and  AMC  should  re-examine  the  role  and 
implementation  of  Manprint  during  systems  acquisition  to  insure  the 
appropriate  use  of  engineering  simulations  as  test-beds  to  identify  and  reduce 
excessive  training  burdens  of  new  and  modified  equipment 


(Management,  EI&MQ,  Methodology)  (Management  Improvement! 


5 .  [IG,  1989]  WARGAMING  ACTIVITIES  IN  THE  DEPARTMENT  OF 
DEFENSE  (REPORT  NO.  89-057)  (MARCH  1989).  WASHINGTON, 
DC:  OFFICE  OF  THE  INSPECTOR  GENERAL,  DEPARTMENT  OF 
DEFENSE. 

•  We  recommend  that  the  Assistant  Secretary  of  Defense  for  Force  Management 
and  Personnel  participate  in  establishing  policies  and  procedures  on  wargaming 
and  similar  activities-including  simulations,  modeling,  and  exercises-that 
define  wargaming  and  clarify  responsibilities  and  relationships  of  DoD 
schools,  colleges,  and  training  centers  involved  in  joint  wargaming  activities. 

(Management.  ET&MO.  Methodology)  [Management  Improvement! 

•  We  recommend  that  the  Director,  Joint  Staff,  prepare  and  staff  a  decision  paper 
for  the  Chairman,  Joint  Chiefs  of  Staff,  to: 

-  Designate  an  office  of  primary  responsibility  within  the  Joint  Staff  to 
establish  policies  and  procedures  on  wargaming  and  similar  activities-- 
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including  simulations,  modeling,  and  exercises-that  define  wargaming 
and  clarify  responsibilities  and  relationships  of  DoD  schools,  colleges, 
and  training  centers  participating  in  joint  wargaming  activities.  These 
policies  and  procedures  should  include  provisions  to  oversee,  monitor, 
and  coordinate  the  acquisition  of  facilities,  computer  systems,  computer 
models,  data  bases,  and  other  developments  related  to  wargaming, 
simulations,  exercises,  and  other  defined  and  specified  operational 
responsibilities  and  missions. 

(Management.  ET&MO.  Methodology!  f  Coordinate! 

--  Discontinue  operations  at  the  Joint  Warfare  Center  and  transfer 
management  oversight  responsibilities  (now  assigned  to  the  Joint  Warfare 
Center)  to  the  Joint  Staff. 

(Investment.  Analysis.  Methodology)  [Management  Improvement! 

Initiate  actions  with  the  Assistant  Secretary  of  the  Army  for  Financial 
Management  and  the  Comptroller  of  the  Air  Force  to  discontinue  fun  ’ing 
the  operations  of  the  Joint  Warfare  Center,  and  adjust  the  Five-Year 
Defense  Plan  accordingly. 

(Investment-  Analysis.  Methodology)  [Management  Improvement! 

•  We  recommend  that  the  Deputy  Chief  of  Naval  Operations  for  Naval  Warfare: 

-  Direct  the  Naval  War  College  to  request  a  waiver  from  the  Joint  Staff  to 
allow  the  development,  testing,  and  analysis  of  operational  war  plans  and 
contingency  plans  for  the  Chief  of  Naval  Operations.  If  denied,  direct  the 
College  to  discontinue  its  support  to  testing  and  analysis  of  plans. 

(Management,  Analysis,  Methodology)  [Management  Improvement! 

-  Stop  constructing  Sensitive  Compartmented  Information  Facilities  at 
Naval  Schools,  colleges,  and  training  centers  when  wargaming  activities 
are  limited  to  education  and  training. 

(InYCStment,  ET&MQ,  Methodology)  [Management  Improvement! 

•  We  recommend  that  the  Air  Force  Deputy  Chief  of  Staff  for  Plans  and 
Operations: 

Direct  the  Air  Force  program  management  office  to  prepare  memorandums 
of  agreement  and  joint  management  plans  to  ensure  that  any  further 
development  of  the  Command  Readiness  Exercise  System  is  coordinated 
with  other  Service  schools,  colleges,  and  training  centers. 

(Management.  ET&MO.  Information  Sharing)  [Management 
Improvement! 
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Direct  the  Air  Force  program  management  office  to  make  requirements  for 
developing  the  Command  Readiness  Exercise  System  consistent  with  the 
Joint  Chiefs  of  Staff  policy  on  the  distribution  of  operational  war  plans 
and  contingency  plans  to  ensure  that,  if  appropriate,  a  waiver  to  this 
policy  is  granted  before  development  of  the  Command  Readiness  Exercise 
System  is  continued. 

(Management,  Analysis,  Methodology)  (Management  Improvement! 

•  We  recommend  that  the  Commander  in  Chief,  U.S.  European  Command: 

-  Direct  the  Warrior  Preparation  Center  to  prepare  an  updated  management 
plan  specifying  the  objectives  of  wargaming  activities,  the  operating 
procedures,  and  the  requirements  for  facilities  and  computer  systems 
necessary  to  effectively  manage  wargaming  activities  there. 

(Management.  ET&MO.  Methodology)  (Management  Improvement! 

Initiate  actions  with  the  Assistant  Secretary  of  the  Army  for  Financial 
Management  and  the  Comptroller  of  the  Air  Force  to  discontinue  the 
$40  million  in  funding  required  by  the  Warrior  Preparation  Center  to 
expand  its  facilities,  to  procure  computer  systems  (hardware  and 
software),  and  to  develop  computer  models.  Adjust  the  Five-Year 
Defense  Plan  accordingly. 

(Investment,  ET&MQ,  Methodology)  (Management  Improvement! 

-  Prepare  and  staff  a  decision  paper  for  the  Secretary  of  Defense  to 
coordinate  the  appropriate  organization  and  command  structure  for  the 
Warrior  Preparation  Center  to  ensure  that  the  U.S.  Army,  Europe;  the 
U.S.  Naval  Forces,  Europe;  and  the  U.S.  Air  Forces  in  Europe  are 
included  in  wargaming  exercises  held  at  the  Center. 

(Management.  ET&MO.  Information  Sharing)  (Management 

Improvement] 

Initiate  actions  to  have  a  memorandum  of  understanding  executed  with  the 
Supreme  Allied  Commander,  Europe,  that  ensures  that  the  U.S. 
Government  is  reimbursed  by  the  North  Atlantic  Treaty  Organization 
nations  that  participate  in  wargaming  exercises  at  the  Warrior  Preparation 
Center. 

(Management,  ET&MQ,  Information  Sharing)  (Management 

Improvement) 
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6 .  [DSB,  1988]  REPORT  OF  THE  DEFENSE  SCIENCE  BOARD  TASK 
FORCE  ON  COMPUTER  APPLICATIONS  TO  TRAINING  AND 
WARGAMING  (MAY  19 88).  WASHINGTON,  DC:  DEFENSE 
SCIENCE  BOARD.  [SPONSORED  BY  THE  UNDER  SECRETARY  OF 
DEFENSE  FOR  RESEARCH  AND  ENGINEERING]  AD-A199  456 

•  Make  joint  simulations  interoperable.  Internet  existing  Service, 
college,  training  center,  and  joint  games  and  simulations  for  education  and 
training.  Evolve  them  to  be  distributed,  so  that  commanders  and  staffs  can 
train  from  their  duty  stations  in  peace  or  wartime.  Use  one  standard  DoD 
communication  protocol,  and  use  cost-effective  communications  hardware  and 
software. 


(Investment.  ET&MO.  Architecture  Development)  fl6ghni£aUmprPYCmgnl1 

•  To  facilitate  the  sharing  of  simulation  data,  create  a  shared  repository,  a  library 
of  automated,  validated  data  descriptions  for  simulation  use.  Make  the  data 
descriptions  available  DoD-wide.  To  ensure  the  quality  and  timeliness  of  data, 
each  data  description  should  be  built  and  maintained  by  the  organization/service 
that  is  most  knowledgeable  about  the  data. 

(Management.  ET&MO.  Information  Sharing)  [Coordinate! 

•  Promote  joint  simulation  usage.  Continue  and  extend  the  involvement 
of  the  most-senior  joint  commanders  in  battle  simulations.  The  CINCs  should 
be  supported  as  they  obtain  required  joint  simulation  capabilities,  particularly 
on-station  capabilities. 

(Management.  ET&MO.  Methodology)  [Application] 

•  Establish  requirements  for  future  capabilities.  Establish  long-term 
joint  training  simulation  requirements.  Document  them  in  JCS  directives  and 
use  these  requirements  to  develop  and  issue  guidance  for  future  system 
designs,  substantive  enhancement  of  current  systems,  development  of  technical 
standards  for  gaming,  training  and  simulation,  and  management  of  a 
simulation-prototype  program. 


(Management.  ET&MO.  Methodology)  [Management  Ii 


cm] 


Establish  a  prototype  program.  Establish  a  continuing  program  to 
demonstrate  exploratory  prototype  simulations,  followed  by  rapid  acquisition 
of  selected  capabilities.  The  program  should  monitor  technology  advances  and 
selectively  build  experimental  prototypes.  The  joint  users  should  be  closely 
involved.  Selected,  prototype-proven  capabilities  should  be  rapidly  fielded. 

(Investment.  R&D.  Advanced  Technology!  [Targeted  R&D1 
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•  Undertake  a  major  joint  training  initiative.  Institutionalize  the 
management  and  budget  in  OJCS  to  oversee:  the  immediate  internetting  of 
existing  assets,  enhanced  joint  simulation  use,  increased  use  of  joint  assets  for 
testing  war  plans  and  joint  doctrine,  creation  of  the  shared  data  repository,  and 
implementation  of  the  rapid  prototype  program.  CJCS  should  coordinate 
spaces,  program  elements,  and  budget  lines— both  joint  and  in  the  Services— to 
arrive  at  more  effective  simulation-based  training  with  less  development 
redundancy. 

(Management.  ET&MO.  Information  Sharing)  fApplicatiOOl 

7 .  [GAO,  1988]  ELECTRONIC  WARFARE:  MULTIPLE  DEVELOPMENT 
OF  COSTLY  THREAT  SIMULATORS  (GAO/NSI AD-88-93) 
(FEBRUARY  1988).  WASHINGTON,  DC:  GENERAL 
ACCOUNTING  OFFICE.  AD- A 190  349 

•  The  Secretary  of  Defense  should: 

-  Assure  that  the  EXCOM  and  Crossbow-S  or  other  appropriate  DoD 
elements  execute  responsibility  and  authority  for  centrally  managing 
simulator  programs  to  provide  for  timely  identification  and  consolidation 
of  simulator  requirements  and  for  disapproval  of  programs  representing 
unwarranted  development 

(Policy,  Analysis,  Methodology)  f  Coordinate] 

-  Require  the  services  to  strengthen  internal  controls  over  simulator 
acquisitions  by  segregating  responsibilities  for  development  testing,  and 
acceptance  of  simulators  as  valid  representations  of  the  threat 

(Management.  Analysis.  Methodology)  [Managcn^nLlrngmyCtilgnt] 

-  Assign  to  an  appropriate  DoD  element  the  responsibility  for  monitoring 
the  quality  of  simulators  acquired  and  participating  in  the  acquisition 
process  as  necessary  to  assure  the  adequacy  of  simulators. 

(Management.  Analysis.  Methodology)  f  Coordinate! 


A- 13 


8.  [ASB,  1988]  ARMY  SCIENCE  BOARD  FINAL  REPORT  OF  THE  AD 
HOC  SUBGROUP  ON  THE  USE  OF  ARMY  COMBAT  MODELS  FOR 
THE  ANALYSIS  AND  TRAINING  OF  JOINT/COMBINED 
OPERATIONS  (JANUARY  1988).  WASHINGTON,  DC:  ARMY 
SCIENCE  BOARD.  [SPONSORED  BY  THE  DEPUTY  UNDER 
SECRETARY  OF  THE  ARMY  FOR  OPERATIONS  RESEARCH] 
AD-B120  937 


Joint  Scenarios  and  Data 

•  Army  should  consider  use  of  multiple  scenarios  for  analysis,  e.g.,  m  Corps 
insertion  in  NORTHAG. 


(Investment.  Analysis.  Methodology)  [Targeted  R&D1 

Scenarios  should  include  dynamics  of  the  onset  of  battle  conditions. 

(Investment.  Analysis.  Methodology)  [Technical  Improvement! 

Establish  joint  USA-USAF  and  USA-USN  analysis  steering  groups  to  guide 
analysis,  model/model  developments,  and  scenarios/databases. 


(Management.  Analysis.  Information  Sharing!  [Technical  Ii 


1 1  i  • )  HI  fl  1 


it] 


Army  should  reconsider  limits  to  data  exchange  with  NATO  allies. 


(Policy.  Analysis.  Information  Sharing)  [Management  Improvement! 


Joint  Campaign  Models 

•  Develop  an  EAC  (Echelon  Above  Corps)  model  by  enhancing  VIC  via  a 
“prototyping”  process. 

(Investment.  Analysis.  Methodology)  [Targeted  R&D1 

•  Replace  assessment  routines  in  training  models  with  those  used  in  analysis 
models. 


(Investment.  ET&MO.  Methodology) 


[Technical  Ii 


MMJL'l 


'eroent] 


Establish  an  Army  models  committee  peer  review  of  Army  models/model 
developments.  Encourage  a  similar  activity  in  USAF,  JCS,  CENTCOM, 
RECOM. 


(Management,  Analysis,  Information  Sharing)  [Management  Improvement] 

Model  developments  should  employ  an  evolutionary,  prototyping  strategy,  not 
a  “bold-leap”  strategy. 

(Management,  MB,  Methodology)  [Technical  Improvement! 
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Joint  Campaign  Analysis  and  Training 

•  DUSA(OR)  should  establish  an  Army  analysis  improvement  program. 
qnvestment.  Analysis.  Methodology)  [Technical  Improvement! 

•  Do  not  use  wargame  training  models  as  analysis  vehicles,  although  they  can 
identify  important  issues  for  analysis. 

(Management.  Analysis.  Methodology)  Management  Improvement! 

•  DUSA(OR)  should  take  a  leadership  position  with  the  JCS-8  regarding  Army 
analysis. 

(Policy.  Analysis.  Methodology)  [Management  Improvement! 


Recommendations  on  Targeted  Issues 

Army  and  Air  Force  Coordination  of  Campaign  Modeling  and 
Analysis 

•  The  VCSA  should  approach  the  VCSAF  to  form  a  joint  analysis  steering  group 
at  the  headquarters  level.  The  group  should  be  responsible  for  stimulating  and 
oversight  of  cooperative  efforts  in  campaign  modeling  and  analyses  where 
deemed  appropriate. 

(Management  Analysis.  Information  Sharing)  r Coordinate! 

•  As  a  first  effort,  the  group  should  formulate  a  joint  study  to  compare  the 
structures,  data,  and  output  of  the  TAC  THUNDER,  FORCEM,  and  VIC 
models  (with  particular  emphasis  on  assumptions,  employment  logic, 
assessment  routines)  and  develop  a  program  to  resolve  major  disparities. 
Further,  it  should  consider  the  development  of  mutually  agreeable  scenarios 
and  a  joint  data  base  for  analyses. 

qnvestment.  Analysis.  Information  Sharing)  (Coordinate! 

Army  and  Marine  Corps/Navy  Coordination  of  Campaign  Modeling 
and  Analysis 

•  Chief  of  Staff  Army  and  Chief  of  Naval  Operations  (CSA/CNO)  should  initiate 
a  relationship  of  the  kind  which  led  to  the  Chief  of  Staff  AF/CSA  agreement  to 
examine  specific  areas  of  mutual  cooperation  in  planning. 

(Management.  Analysis.  Information  Sharing!  fCoordinatel 

•  As  a  first  effort  CSA/CNO  should  develop  several  test  scenarios  of  joint 
warfare,  one  examining  a  typical  contingency  situation  where  Marines  and 
Army  units  operate  together  and  another  where  Army  electronic  counter- 


measures  (ECM),  electronic  counter-countermeasures  (ECCM)  battlefield 
needs  are  provided  by  the  Navy  until  Army  assets  arrive. 

(investment.  Analysis.  Information  Sharing)  fApplicatiopI 

•  VCSA  should  approach  Vice  CNO  (VCNO)  to  form  a  HQ-level  joint  analysis 

steering  group  similar  to  that  recommended  for  the  VCSA  and  VCAF. 
Additionally,  the  VCSA  and  VCNO  should  establish  a  joint  ASB,  Naval 
Research  Advisory  Council  Task  Team  to  recommend  the  appropriate  level  of 
USN/USA  cooperation  in  analysis. 

(Management.  Analysis.  Information  Sharing)  [Management  Improvement! 


Army  and  Allied  Forces  Coordination  of  Campaign  Modeling  and 
Analysis 

•  The  Army  should  improve  its  consideration  of  combined  forces  in  its  analysis 
of  U.S.  defense  issues  and  of  combined  issues.  It  could  begin  by  learning 
more  about  and  (where  useful)  exploiting  analyses  performed  by  SHAPE 
(Supreme  Headquarters  Allied  Powers,  Europe)  Technical  Center  (STC). 
Adequate  consideration  of  combined  issues  will  require  the  U.S.  to  take  a 
proactive  position  to  open  up  the  exchange  of  data  with  the  allies,  particularly 
weapon  system  performance  data. 

(Investment.  Analysis.  InformationSharing)  [Technical  Improvement! 


Implications  to  the  Army  of  the  JCS  Reorganization  and  Packard 
Commission 

•  In  view  of  the  expanded  charter  of  the  JCS,  DUSA(OR)  should  take  a 
leadership  position  for  the  Army  to  ensure  the  continuation  of  a  strong  liaison 
with  J-8  during  both  the  formative  period  and  the  long  term. 

(Management  Analysis.  Information  Sharing)  fCwrdlnatgl 

•  It  is  recommended  that  the  Joint  Analysis  Steering  Group  should  take  the  lead 
in  fostering  the  development  of  joint  inputs  for  submission  to  J-8,  not  only  to 
provide  assistance  but  also  to  share  confidence  that  joint  Army/Air  Force  views 
are  considered. 

(Management  Analysis.  Information  Sharing)  [Management  Improvement! 

•  Quality  Army  personnel  should  be  made  available  for  assignment  to  J-8  to 
ensure  that  solid  Army  experience  is  provided  and  that  ground-force 
considerations  are  properly  represented.  Specifically,  it  is  recommended  that 
the  ORSA  Advisory  Committee  be  charged  with  the  responsibility  for  seeing  to 
it  that  premium  Army  analysts  are  channeled  into  J-8.  The  Committee  should 


also  carefully  monitor  the  assignment  of  Army  analysts  to  the  staffs  of  the 
Unified  Commands. 


(Management  Analysis.  Information  Sharing)  [Management  Improvement! 


Coupling  the  CINCs'  Needs 

•  The  Army  analysis  community  should  be  attuned  to  the  thinking  of  CINCs  and 

aware  of  current  Operations  Plans  (OPLANS)  so  that  this  thinking  can  be 
embodied  in  the  future  scenarios,  threats,  and  data  which  are  used  in  Army 
analyses.  This  should  be  coordinated  with  JCS  and  the  MAPP. 


(Management  Analysis.  Information  Sharing)  [Technical  Improvement! 


DUSA(OR)  should  endeavor  to  have  the  Army  analysis  community  participate 
with  J-8  JCS  in  supporting  the  CINCs  needs. 


(Management  Analysis.  Information  Sharing)  [Management  Ii 


I] 


Adequacy  of  Scenarios  for  Joint/Combined  Analysis  and  Training 

•  The  DUSA(OR)  should  form  an  ad  hoc  team  to  review  this  issue  (adequacy  of 
joint  scenarios)  and  suggest  appropriate  scenarios  and  a  process  to  ensure  their 
consideration. 


(Investment.  Analysis.  Methodology) 


[Technical  Imi 


1] 


Campaign  Models  for  Analysis  of  Joint  Operations 

•  Army  should  develop  a  multi -corps-to- Army-group  automated  simulation. 
TRAC  should  enhance  VIC  for  this  purpose  using  a  prototyping  process. 
Then  use  the  model  for  analysis  of  most  joint  issues. 

(Investment.  Analysis.  Methodology)  [Application! 

•  Enhance  representation  of  tactical  air  processes  in  FORCEM.  Expand  scope 
where  necessary.  Use  for  analysis  of  theater-level,  force  structure,  logistics, 
and  major  resource  allocation  joint  issues. 

(Investment.  Analysis.  Methodology)  [Targeted  RAD! 

•  Wargame  models  are  best  used  in  broad  identification  of  issues  and 
development  of  insights.  Wargame  models  such  as  MTM,  JTLS,  and  JESS 
should  not  be  used  for  the  evaluation  of  alternative  solutions  or  sensitivity 
analyses. 


(Management.  Analysis.  Methodology)  [Management  Ii 
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Adequacy  of  Data  for  Joint/Combined  Analysis 

•  Require  all  Army  models  to  use,  to  the  maximum  extent  possible,  measurable 
data  generated  by  Army  and  Air  Force  laboratories  for  the  assessment  routines. 
DUSA(OR)  should  audit  compliance  with  this  recommendation. 


(Management  Analysis.  Information  Sharing)  [Technical  h 


MW  lU'A-i  MMI 


Army  Campaign  Analysis  Process 

•  The  analysis  process  of  studying  all  the  component  pans  of  the  simulated 
campaign  results  in  detail  to  learn  why  the  more  macro  results  occurred  and  to 
verify  that  they  are  correct  is  not  a  school-learned  ability.  The  Army  must 
recognize  this  training  requirement  and  make  a  conscious  effort  to  provide  this 
training  to  its  analysis  staffs  through  the  formation  and  implementation  of  an 
"Army  Analysis  Improvement  Program"  (AAIP)  using  senior  level  analysts  as 
trainers. 

-  Establish  an  Army  analysis  improvement  program. 

(Management.  ET&MQ.  Methodology)  [Management  Improvement! 

-  ODUSA(OR)  select  5-10  studies  annually  for  close  personal  participation 
resulting  in  mentoring  and  technical  review. 

(Investment,  ET&MQ.  Methodology)  [Management  Improvement! 

•  As  noted,  it  takes  many  years  to  learn  how  to  verify  the  correctness  of  a 
simulated  campaign  run  and  to  extract  meaningful  insights  and  trends  from 
multiple  runs.  Maintaining  a  small  number  of  joint  analysis  teams  doing 
continual  analysis  of  airland  campaign  issues  (rather  than  forming  a  new  team 
for  each  study  question  )  would  facilitate  this  learning.  The  Army  should 
formulate  and  implement  a  research  program  to  see  if  an  "Intelligent  Analyzer” 
expen  system  can  be  developed  which  captures  the  analysis  process  used  by 
recognized  senior  experienced  analysts  and  could  be  used  in  training  less 
experienced  ones  and  possibly  suppon  the  analysis  activity. 

--  Require  analyses  be  performed  by  teams  comprised  of  analysts,  modelers 
who  know  the  details  of  the  model  used,  data  specialists,  and  relevant 
military  operational  specialists. 

(Management,  Analysis,  Methodology)  [Management  improvement! 

Have  TRADOC  establish  a  proactive  cell  of  analysts  and  operations/ 
doctrine  experts  to  perform  continuous  analysis  of  joint/combined  tactics, 
operations,  and  doctrine.  This  cell  should  be  designated  as  the  Army's 
overall  agency  for  coordination  of  joint  operations  in  Army  analysis. 


(Investment,  Analysis,  Methodology)  (Management 


Initiate  research  to  develop  an  "intelligent  analyzer"  expert  system  and 
"analytic  analog”  campaign  models. 

(Investment.  Analysis.  Advanced  Technology)  fTargeted  R&D1 


•  Parametric  analysis  is  an  integral  part  of  any  Army  analysis  project. 
Recognizing  that  time  and  resource  constraints  may  preclude  the  number  of 
parametric  variations  one  would  like  in  a  study,  the  Army  should  initiate  a 
program  to  develop  and  test  "analytic  analog"  campaign  models  that  can  mimic 
the  more  detailed  corps  and  Echelons  Above  Corps  (EAC)  ones  for 
interpolative  sensitivity  analysis.  (Examples  of  this  concept  include  the 
COMAN,  BLDM,  AFAADS-GUN,  MACRO,  and  other  analytic  models.) 

-  Require  parametric  analysis  in  study  directives. 


(Management  Analysis.  Methodology! 


flscluiigal.il 


-  Require  briefings  and  reports  to  highlight  key  variables,  assumptions, 
FTC  that  are  "high  drivers." 


(Management,  Analysis,  Methodology)  Management  Improvement! 


Campaign  Models  for  Training  in  Joint  and  Allied  Operations 

v 

•  Model-based  training  may  provide  valuable  practice  and  experience  in 
warfighting  for  commanders  and  staffs.  It  should  continue  to  be  resourced. 
However,  it  is  imperative  that  the  simulation  algorithms  be  significantly 
improved.  The  assessment  routines  in  MTM,  JTLS,  and  JESS  should  be 
replaced  by  those  used  by  the  Army  corps-level  analysis  models,  perhaps  with 
some  analytic  modifications  to  reduce  run  times. 

(Investment,  ET&MQ,  Methodology)  fTechnical  Improvement! 

•  The  Army  should  expedite  the  process  of  providing  a  responsive,  EAC  training 
model  for  use  by  senior  commanders. 

(Investment.  EI&MQ,  Methodology)  r  Application! 


Credibility  of  Some  Campaign  Models 

*  The  DUSA-OR  should  form  a  DA  level  group  comprised  of  senior  experienced 
professionals  to  structure  and  implement  procedures  for  peer  review  and 
scrutiny  of  Army  campaign  models  (and  possibly  campaign  analyses).  The 
Army  should  strongly  encourage  the  establishment  of  similar  groups  in  DoD 
organizations  outside  the  Army-especially  the  Air  Force  and  JCS-who 
routinely  use  simulation  models  for  analysis  and  training  which  involve  Army 
forces. 

(Management.  Analysis.  Methodology!  [Coordinate! 
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•  Exposure  to  the  professional  community  is  a  generic  mechanism  for  peer 
review  and  scrutiny  of  scientific/technical  work.  Models  used  in  Army  joint 
studies  should  be  used,  and  be  technically  understood ,  by  multiple  agencies. 
All  models  used  for  analysis  or  training  of  Army  forces  should  use  or  have  a 
direct  audit  trail  to  objective  measurable  data  generated  by  Army  and  Air  Force 
laboratories  for  their  assessment  routines. 

(Management  Analysis.  Information  Sharing)  Management  Improvement! 


Coordination  of  Army  Campaign  Modeling  Efforts 


Army  models  committee  should  coordinate  and  review  all  the  Army's  ongoing 
campaign  modeling  EFF  efforts. 


(Management  Analysis.  Information  Sharing)  Management  Improvement! 

Model  development  should  be  evolutionary  and  iterative  in  conjunction  with 
studies.  Do  not  repeat  AMD3  "bold-leap"  strategy. 


Management  R&D.  Methodology) 


[Technical  Ii 


I] 


VCSA  and  DUSA(OR)  should  ensure  that  AMIP  funding  is  maintained. 


Management.  Analysis.  Methodology)  (Technical  Ii 


I] 


•  AMC  should  expand  the  Army  Model  Management  Office's  (AMMO)  mission 
to  include  execution  of  joint/combined  modeling  developments. 

(Management  Analysis,  Methodology)  Management  improvement! 


Opportunity  for  Senior  Officer  Exercises 

•  Adopt  a  primary,  simulation-based  EAC  senior  officer  training  model. 
Candidates  include  VIC  (enhanced)  (Issue  #7)  or  more  rapid  versions  of  JTLS, 
JESS  (expanded)  and  FORGE. 

(Investment  ET&MO.  Methodology)  (Application! 

•  WPC  concept  should  be  fostered  in  unified  commands  and  senior  service 
colleges. 

(Management,  ET&MO.  Methodology)  (Technical  improvement! 

•  Army  should  fully  support  Air  University  "Joint  Flag  Officer  War  Fighting 
Course,"  and  examine  a  similar  capability. 


(Investment  ET&MO.  Methodology)  f  Application! 


Available  Technology  Opportunities  for  Improvement  of  Campaign 
Modeling  and  Analysis 

•  AMC  should  expedite  efforts  to  simplify  and  speed  up  process  of  developing 
computer  model  input/output  through  use  of  model  subset  libraries  and 
relational  databases. 

(Investment.  Analysis.  Methodology)  (Technical  Improvement! 

•  AMC,  as  a  matter  of  priority,  should  procure  contemporary  high-speed  data 
processing  equipment  for  its  analytic  agencies—failing  in  this,  authorize 
contemporary  equipment  rental. 

(Investment.  Analysis.  Advanced  Technology)  fTechnical  Improvement! 

•  AMC  should  focus  on  the  adaptation  of  commercial  graphics  improvements  to 
aid  input/output  data  interpretation. 

(Investment.  Analysis.  Methodology)  fTechnical  Improvement! 

•  AMC  should  evaluate  NATO  Panel  VII  concerns  and,  if  appropriate,  develop 
distributed  wargames/models  to  allow  remote  participation  by  operational 
commands  and  non-collocated  analysts. 

(Investment.  Analysis.  Information  Sharing)  [Targeted  R&P1 

•  AMC  should  establish  properly  resourced  and  integrated  artificial  intelligence 
cells  at  analytic  agencies,  to  include  LANL. 

(Investment.  Analysis.  Advanced  Technology)  [Targeted.  R&B1 

•  Within  the  context  of  the  modeling  R&D  Plan,  Army  Model  Management 
Office  (AMMO)  should  investigate  artificial  intelligence  to  enhance  pre-/post- 
processors,  enhance  sensitivity  analysis,  better  represent  red  and  blue 
commands,  and  provide  modular  staffs  in  models. 

(Investment.  Analysis.  Advanced  Technology)  [Targeted JR&B1 

•  HQDA  AI  Center  address  the  apparent  lack  of  transportability  in  diverse  AI 
hardware  and  software. 

(Management.  Analysis.  Information  Sharing)  [Technical  Improvement! 

Basic  Research  Program  in  Campaign  Modeling  and  Analysis 

•  AMC  should  conduct  a  model  deficiency  analysis  and  prioritization. 

(Investment,  R&D,  Advanccdjcchnplogy)  fM&S  Development! 

•  AMC  should  conduct  a  modeling-related  technology  forecast 

(Investment.  R&D.  Advanced  Technology)  fM&S  Development! 


A-21 


•  AMMO  should  design  the  research  program  resulting  from  this  forecast  and 
deficiency  analysis. 

(Management.  R&D.  Advanced  Technology)  [Targeted  R-&P1 

•  AMC  should  designate  AMMO  as  the  manager  of  this  research  program,  with 
AMC  guidance  and  personnel  augmentation. 

(Management.  R&D.  Advanced  Technology)  rTargeted  R&D1 

•  AMC  should  expand  AMMO's  mission,  to  include  coordinating  research  with 
external  agencies,  such  as  Defense  Advanced  Research  Project  Agency 
(DARPA),  JCS,  RAND,  national  labs  and  academia. 

(Management,  R&D.  Information  Sharing)  [Management  Improvement! 

•  DUSA(OR)  should  recommend  that  DARPA  host  a  semi-annual  joint  modeling 
research  seminar. 

(Management.  R&D.  Information  Sharing!  [Management  Improvement! 

•  VCSA  and  DUSA(OR)  should  ensure  that  AMIP  and  SIMTECH  funding  is 
maintained  or  expanded. 

(Management.  R&D.  Methodology)  [Technical  Improvement! 


[GAO,  1987]  DOD  SIMULATIONS:  IMPROVED  ASSESSMENT 
PROCEDURES  WOULD  INCREASE  THE  CREDIBILITY  OF 
RESULTS  (GAO/PEMD-88-3)  (DECEMBER  1987).  WASHINGTON, 
DC:  GENERAL  ACCOUNTING  OFFICE.  AD-A191  503 


•  GAO  recommends  that  the  Secretary  of  the  Department  of  Defense  adopt  or 
develop  and  implement  guidance  on  producing,  validating,  documenting, 
managing,  maintaining,  using,  and  reporting  simulations  of  weapon-system 
effectiveness.  This  guidance  should  include  a  way  of  routinely  providing 
reviews  of  a  simulation's  credibility  and,  in  this  way,  identifying  problems  that 
should  be  resolved.  The  Secretary  should  explore  requiring  that  a  statement 
regarding  validation  efforts  accompany  simulation  results. 


(Management.  Analysis.  Methodology)  [M&S  Devcl 


•i'JflHI 


I] 


GAO  also  recommends  that  the  Secretary  of  the  Department  of  Defense  direct 
the  agencies  responsible  for  managing  the  ADAGE,  CARMONETTE,  and 
COMO  III  models  to  explore  the  feasibility  of  correcting  the  limitations  GAO 
has  identified,  especially  the  limitations  in  validation. 


(Management.  Analysis.  Methodology)  [Technical  Improvement] 


10.  [DSB,  1986]  CONFLICT  ENVIRONMENT  TASK  FORCE 

(IMPLICATIONS  OF  THIRD  WORLD  URBAN  INVOLVEMENT) 
(MAY  1986).  WASHINGTON,  DC:  DEFENSE  SCIENCE  BOARD. 
[SPONSORED  BY  THE  UNDER  SECRETARY  OF  DEFENSE  FOR 
RESEARCH  AND  ENGINEERING]  AD-A171  677 


Longer  Term  Mission  Preparedness 

•  The  Corps  of  Engineers  should  conduct  and  contract  for  games,  simulations, 
and  exercises  to  clarify  and  identify  requirements. 

(Investment.  Analysis.  Methodology)  [Application! 

•  The  Secretary  of  Defense  and  the  Chairman  of  the  Joint  Chiefs  of  Staff  should: 

-  Institutionalize  "from  the  beginning"  intelligence  community  participation 
in  Third  World  operations  planning  and  pertinent  games,  simulations,  and 
exercises. 

(Management,  EE&MQ,  Infoimation-Sharins)  [Technical  improvement! 

-  Task  the  Services  to  institutionalize  on-going  education  and  training, 
simulations,  and  games  on  Third  World  conflicts  with  urban  control  and 
management  aspects  in  the  Service  academies  and  the  command  and  staff 
colleges. 

(Investment.  ET&MO.  Methodology)  [Application] 

11.  [ASB,  1985A]  ARMY  SCIENCE  BOARD  FINAL  REPORT  OF  THE 
1985  SUMMER  STUDY  ON  TRAINING  AND  TRAINING 
TECHNOLOGY-APPLICATIONS  FOR  AIRLAND  BATTLE  AND 
FUTURE  CONCEPTS  (DECEMBER  1985).  WASHINGTON,  DC: 
ARMY  SCIENCE  BOARD.  [SPONSORED  BY  THE  DEPUTY  CHIEF 
OF  STAFF  OF  THE  ARMY  FOR  OPERATIONS]  AD-B101  (MOL 

Battle  Simulation 

•  TRADOC  should  sustain  training  battle  simulation  improvement  programs. 
Align  training  simulation  models  more  closely  with  model  structures  used  for 
analysis  (i.e.,  use  of  detailed  process  descriptions). 

(InYCStmsm,  EI&MQ,  Methodology)  fTechnical  Improvement! 

•  TRADOC  should  incorporate  new  technologies  to  reduce  controller 
requirements  and  increase  simulation  portability. 

(Investment.  EI&MO.  Advanced  Technology!  [Targeted  R&D1 
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•  ODCSOPS  should  fund  and  AMC/TRADOC  accelerate  SIMNET  development 
and  explore  potential  to  connect  SIMNET  with  larger  Battle  Simulations. 

(Investment  ET&MO.  Methodoloev)  fTareeted  R&D1 

•  TRADOC  should  increase  the  emphasis  on  managing  the  conduct  of  unit 
training  in  Army  Officer/NCO  institutional  training. 

(Management.  ET&MO.  Methodology)  rAnplicationl 

Leadership 

•  ODCSOPS/TRADOC  develop  and  make  available  to  the  General  Officer  a 
specifically  tailored  computerized  warfighting  game  providing  the  opportunity 
for  personal  rapid  experimentation  in  the  operational  art/tactics  of  AirLand 
Battle  (ALB)  doctrine. 

(Investment  ET&MQ.  Methodology)  (Application! 

Validation  of  Training  System  Components 

•  TRADOC  should  establish  a  panel  of  training  technologists  to  identify  training 
DSM&Ps  (Devices,  Simulations,  Methods,  and  Practices)  requiring  validation 
through  controlled  testing  or  other  techniques;  then  fully  support  such 
evaluations. 

(Management,  ET&MQ.  Methodology)  (Technical  Improvement! 

•  Agencies  responsible  for  the  design,  conduct,  and  reporting  of  validation  tests 
should  be  adequately  staffed,  and  their  efforts  subjected  to  independent  review 
by  peers  responsible  to  the  DA  Staff. 

(Management.  ET&MQ.  Methodology)  (Management  Improvement! 

12.  [DSB,  1985]  IMPROVING  THE  ACQUISITION  MANAGEMENT 
PROCESS  FOR  CONVENTIONAL  MUNITIONS ,  FOCUSING  ON 
GROUND  ATTACK  MUNITIONS  (U)  (NOVEMBER  1985). 
WASHINGTON,  DC:  DEFENSE  SCIENCE  BOARD.  [SPONSORED 
BY  THE  UNDER  SECRETARY  OF  DEFENSE  FOR  RESEARCH  AND 
ENGINEERING]  (SECRET) i  AD-C037  912 

•  Continue  improving  the  analytical  models  used  for  the  determination  of 
conventional  munitions  acquisition  objectives. 

(Investment,  Analysis,  Methodology)  (Application! 


l 


Although  this  report  is  classified,  both  its  title  and  the  recommendation  listed  here  are  unclassified. 
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13.  [ASB,  1985B]  ARMY  SCIENCE  BOARD  FINAL  REPORT  OF  THE 
1985  SUMMER  STUDY  ON  MANNING  IMPLICATIONS  OF 
LOGISTICS  SUPPORT  FOR  AIRLAND  BATTLE  (OCTOBER  1985). 
WASHINGTON,  DC:  ARMY  SCIENCE  BOARD.  [SPONSORED  BY 
THE  DEPUTY  CHIEF  OF  STAFF  OF  THE  ARMY  FOR  PERSONNEL 
AND  THE  DEPUTY  CHIEF  OF  STAFF  OF  THE  ARMY  FOR 
LOGISTICS)  AD-A163  385 


Data  Bases/Modeling 

Adequate  and  accurate  technical  documentation,  data  bases,  and  logistics  models  are 
required  to  define  and  subsequently  meet  AirLand  battle  combat  service  support  logistics 
requirements. 

•  TRADOC/AMC  should  institute  procedures  to  ensure  adequate  funding  and  to 
discipline  the  requirement  for  RAM/ILS  programs  during  development  of  new 
systems,  product  improvement  programs,  and  acquisition  of  non* 
developmental  items  (NDI). 

(Management.  P&L.  Methodology)  [Technical  Improvement! 

•  TRADOC/AMC  should  develop  and  institute  a  streamlined  logistics  data  base 
program,  oriented  toward  the  needs  of  CSS  force  structure  planners  with  a 
payoff  for  the  user. 

(Investment,  P&L.  Methodology)  [Targeted  MB] 

•  AMC,  emphasizing  simplicity,  should  standardize  and  rigorously  discipline 
Held  logistics  data  collection  efforts,  e.g.,  redefine  the  data  formats  for  LSA 
and  SDC  as  standards. 

(Management,  E&L,  Methodology)  [Technical  Improvement! 

•  DCSOPS  should  develop  CSS  force  structure  planning  models  which  are 
structured  to  reflect  specific,  dynamic,  operational  scenarios,  and  DCSLOG 
account  for  the  productivity  gains  available  through  workload  aggregation. 

(Investment,  P&L,  Methodology)  [Targeted  R&P1 
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14.  [ASB,  1983]  ARMY  SCIENCE  BOARD  (ASB)  1983  SUMMER 
STUDY  ON  THE  FUTURE  DEVELOPMENT  GOAL  (NOVEMBER 
1983).  WASHINGTON,  DC:  ARMY  SCIENCE  BOARD. 
[SPONSORED  BY  THE  ASSISTANT  SECRETARY  OF  THE  ARMY 
FOR  RESEARCH,  DEVELOPMENT,  AND  ACQUISITION] 

AD-A135  819 

Combined  Arms  Training  Strategy 

•  TRADOC  should  devise  a  training  strategy  to  develop  leaders  and  units  for 
airland  battle  (ALB). 

--  Publish  specific  combined  arms  tactics  (to  include  CS,  CSS,  and  other 
services)  to  implement  ALB  doctrine  at  cotps,  division,  and  brigade. 

Identify  training  facility  requirements  and  needed  improvements. 

Identify  requirements  for  improved  training  games,  simulations,  devices, 
etc. 

(Management.  ET&MO.  Methodology)  r  Application] 

15.  [JDL,  1983]  REPORT  OF  THE  JOINT  DIRECTORS  OF 
LABORATORIES  TECHNOLOGY  INITIATIVE  PANEL  FOR 
TRAINING  AND  SIMULATION  (JULY  1983).  (AFHRL-TP-83-30). 
BROOKS  AFB:  AIR  FORCE  HUMAN  RESOURCES  LABORATORY. 
AD-B082  764 

•  It  is  recommended  that  the  JDL  direct  implementation  of  a  joint  program  with 
Air  Force  lead  to  establish  a  Joint  Services  Visual  Simulation  Research  (VSR) 
Program  for  initiation  of  university  research  efforts  to  stimulate  innovations  in 
visual/imaging  simulation  technology,  with  possible  new  breakthroughs  and 
new  architectures  in  training  simulators. 

(Investment.  R&D,  Advanced  Technology)  [Targeted  R&D1 

•  It  is  recommended  that  the  Services  continue  to  support  the  Hermann  model  of 
a  tri-Service  training  and  simulation  R&D  center  of  excellence  with  close 
university  relations  in  Orlando,  FL. 

(Investment.  R&D.  Advanced  Technology]  [Coordinate! 

•  It  is  recommended  that  the  JDL  direct  guidance  to  their  respective  Service 
laboratories: 

To  develop  vertically  integrated  technology  demonstrations  that  are  field 
deployable  for  collective  readiness  training  in  units,  especially  in 
command-and -control  training  and  in  wargaming 


(Investment.  R&D.  Methodology]  [Targeted  R&D1 


-  To  appoint  ad  hoc  teams  to  identify  and  assess  possible  applications  of 
existing  training  and  simulation  technology  to  selected  operational  training 
programs. 

(Management.  R&D.  Methodology)  [Application! 

16.  [GAO,  1983]  BETTER  PLANNING  AND  MANAGEMENT  OF 

THREAT  SIMULATORS  AND  AERIAL  TARGETS  IS  CRUCIAL  TO 
EFFECTIVE  WEAPON  SYSTEMS  PERFORMANCE  (GAO/MASAD- 
83-27)  (JUNE  1983).  WASHINGTON,  DC:  GENERAL 
ACCOUNTING  OFFICE.  AD-A129  851 

•  The  Secretary  of  Defense  should: 

--  Require  the  Director,  DT&E,  to  enforce  existing  requirements  for  the 
preparation  and  approval  of  weapon  system  TEMPs  (Test  and  Evaluation 
Master  Plans)  before  the  demonstration/validation  and  subsequent 
decision  milestones. 

(Policy,  T&E.  Methodology)  fM&S  Development! 

—  Require  predemonstration/validation  phase  TEMPs  to  state  whether  or  not 
test  resource  requirements  are  available,  and  outline  what  actions  have 
been  or  need  to  be  taken  to  develop  or  acquire  those  not  available.  In 
addition,  the  effect  of  being  unable  to  test  against  the  full  threat  spectrum 
should  be  clearly  identified. 

(Management,  T&E.  Methodology)  IM&S  Development! 

-  Require  operational  test  and  evaluation  agencies  to  state  in  the  initial 
TEMP  their  ability  (or  inability)  to  adequately  assess  a  weapon's 
operational  suitability  and  effectiveness,  given  currently  available  test 
resources.  The  adequacy  of  test  resources  and  the  effect  of  inadequate  or 
incomplete  testing  should  be  clearly  spelled  out. 

(Management,  l&E,  Methodology)  rM&S  Development! 

Require  electronic  warfare  and  aerial  target  test  resource  developers  to 
work  closely  with  the  weapons  developers  and  test  agencies  during  early 
test  planning  to  identify  the  critical  test  resources  needed  to  fully  assess 
weapon  system  effectiveness. 

(Management.  T&E.  Methodology)  fM&S  Development! 

Transfer  Navy  and  Air  Force  threat  simulator  and  aerial  target  acquisition 
responsibility  to  an  organization  separate  from  the  weapon  systems 
development  activity.  The  gaining  organization  should  have,  as  does  the 
Army's  threat  simulator  organization,  the  independence,  authority. 
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responsibility,  and  funds  to  ensure  the  successful  acquisition  of  test 
resources. 

(Management.  T&E.  Methodology)  [Coordinate] 

-  Establish  a  joint-Service  threat  simulator  and  aerial  target  improvement 
program  to  identify,  time  phase,  and  prioritize  DoD-wide  test  resource 
deficiencies;  and  start  development  of  the  resources  necessary  to  match  the 
test  capability  with  current  requirements. 

(Investment.  T&E.  Information  Sharing)  f Coordinate] 

--  Initiate  a  review  of  intelligence  support  to  identify  the  underlying  causes 
and  to  solve  the  problems  of  inadequate  support  to  the  threat  simulator 
development  community.  In  particular,  the  appropriateness  of  Defense 
Intelligence  Agency  assignments  to  the  Service  intelligence  organizations 
and  the  capability  of  those  organizations  to  support  both  weapons 
designers  and  the  test  community  should  be  examined  and  changes  made 
where  appropriate. 

(Investment.  T&E.  Information  Sharing)  Management  Improvement! 

17.  [DSB,  1982A]  REPORT  OF  THE  DEFENSE  SCIENCE  BOARD  1982 
SUMMER  STUDY  PANEL  ON  TRAINING  AND  TRAINING 
TECHNOLOGY  (NOVEMBER  1982).  WASHINGTON,  DC: 

DEFENSE  SCIENCE  BOARD.  [SPONSORED  BY  THE  UNDER 
SECRETARY  OF  DEFENSE  FOR  RESEARCH  AND  ENGINEERING] 
AD-A127  596 

Organization  and  Management  Section 

•  Recommendation:  Increase  use  of  analytical  methods  to  (a)  assess/project 
impact  of  manpower  pool  on  new  weapons  systems  and  (b)  identify  where 
training  may  increase  skills/performance  of  recruits  to  meet  system  needs,  do 
not  wait  for  more  analysis/assessment.  There  are  enough  data  to  proceed  now. 

•  Action:  ASD(MRA&L)  establish  policy  that  will  require  design  tradeoffs  and 
contractor  assessments  early  in  the  weapon  system  development  phases  to 
identify  their  impact  on  weapon  system  design  and  skill  performance 
requirements.  Manpower  and  training  projections  be  used  to  identify  impact 
cm  weapon  system  design  and  skill  performance  requirements. 

(Management,  EE&MQ,  Methodology)  [Technical  improvement] 
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Planning  and  Application  Section 

•  Recommendation:  Support  research,  development,  and  use  of  war  games  that 
provide  intelligent  adversaries  and  realistic  conditions  to  promote  effective 
combat  leadership  training. 

•  Action:  USDRE  provide  guidance  to  Military  Departments  to  increase 
emphasis  for  research,  development  and  implementation  of  campaign  battle  and 
engagement  simulation  for  purposes  of  leadership  training. 

(Investment.  ET&MO.  Advanced  Technology)  fTargeted  R&D1 
Technology  Section 

•  Recommendation:  Increase  exploration  and  use  of  current/advanced 
technology  devices  (e.g.,  arcade-like  games  and  simulations)  to  motivate  and 
teach  functional  skills. 

•  Action:  USDRE  provide  guidance  to  Military  Departments  to  increase  funds  to 
explore  use  of  such  devices.  Funds  to  come  from  other  than  Personnel  and 
Training  R&D  programs. 

(Investment-  ET&MO.  Advanced  Technology^  [Targeted  R&D1 

18.  [DSB,  1982B1  DEFENSE  SCIENCE  BOARD  SUMMER  STUDY 
(1982)  PANEL  ON  NEW  WEAPONS  CONCEPTS  (NOVEMBER 
1982).  WASHINGTON,  DC:  DEFENSE  SCIENCE  BOARD. 
[SPONSORED  BY  THE  UNDER  SECRETARY  OF  DEFENSE  FOR 
RESEARCH  AND  ENGINEERING]  AD-C030  727 

•  USDRE  require  the  Services  to  make  more  extensive  use  of  models  and 
simulations  in  evaluating  the  military  worth  and  costs  of  systems— "simulate 
before  decide.” 

(Management.  Analysis,  Methodology)  [Application! 

19.  [DDRE,  1982]  USDRE  INDEPENDENT  REVIEW  OF  DOD 
LABORATORIES  (MARCH  1982).  WASHINGTON,  DC:  OFFICE 
OF  THE  UNDER  SECRETARY  OF  DEFENSE  FOR  RESEARCH 
AND  ENGINEERING.  AD-A118  006 

•  The  creation  of  a  private  sector  organization  as  a  core  work  force  for  the 
development  of  long-term,  professional  quality  in  simulation  techniques. 

(Management,  R&D,  Methodology)  [Coordinate! 
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20.  [ASB,  1981]  ARMY  SCIENCE  BOARD  AD  HOC  SUBGROUP  ON 
TESTING  OF  ELECTRONIC  SYSTEMS  (SEPTEMBER  1981). 
WASHINGTON,  DC:  ARMY  SCIENCE  BOARD.  [SPONSORED  BY 
THE  ASSISTANT  SECRETARY  OF  THE  ARMY  FOR  RESEARCH, 
DEVELOPMENT,  AND  ACQUISITION]  AD-A123  265 

Systems  Development 

•  Additional  effort  should  be  devoted  to  the  concept  definition/concept 
evaluation/advanced  development  phases  of  system  development;  additional 
consideration  should  be  given  to  early  system  simulation  and  to  tradeoffs 
among  performance  and  reliability/availability/maintainability;  in  this 
connection.  Army  in-house  capability  as  "wise  buyers"  should  be  improved. 

(Management,  I&E,  Methodology!  r  Application! 

Software  Design  and  Testing 

•  Based  on  the  system  specifications,  and  with  flexibility  in  agreed  areas, 
automated,  computer-based  test  tools  should  be  developed  to  drive  (via 
simulation  and  stimulation)  the  engineering  and  initial  production  models  of 
software-intensive  systems;  only  in  this  way  can  operational  environments  be 
suitably  represented  in  a  reproducible  fashion. 

Qaycstmem,  T&E,  Methodology)  rM&S  Development! 

•  To  facilitate  cost-effective  software  testing  with  results  that  can  be  uniformly 
interpreted  and  "graded,"  a  common  library  of  software  verification  and 
validation  tools  should  be  developed  and  used  on  an  Army-wide  basis;  the 
Army  should  recognize  an  opportunity  to  provide  (DoD)  leadership  in  this 
regard. 

(Investment.  T&E.  Information  Sharing)  [Technical  Improvement! 

Time  to  Develop  and  Deploy 

•  It  is  suggested  that  consideration  be  given  to  a  radical  change  in  the 
development/testing  process,  in  recognition  of  the  special  characteristics  of 
software-intensive  systems;  that  the  computer-based  test  tools  required  to 
represent  the  test  (tactical)  environment  be  provided  by  a  contractor  other  than 
the  system  development  contractor,  in  parallel  with  system  development  In 
this  approach,  the  testing/user  activities  should  participate  in  the  test  contractor 
design  reviews— and  should  be  required  to  quantify  and  document  test 
requirements. 


(Efilicy.  T&E,  Methodology)  fM&S  Development! 


•  The  indicated  test  drivers  (environment  simulators)  should  be  developed  for 
particular  programs;  however,  they  can  be  appropriately  integrated  into  the 
plans  for  testing  at  various  facilities. 

(Management.  T&E.  Information  Sharing)  fM&S  Development! 

•  The  development  of  the  test  drivers  should  be  in  accordance  with  the 
disciplines  previously  outlined  for  software  development  and  testing. 

(Investment.  T&E.  Methodology)  IM&S  Development! 

Interoperability  Testing 

•  More  extensive  over-all  planning  should  be  carried  out;  and  considered 
decisions  should  be  made  relative  to  the  omission  of  testing  for  reasons  of 
complexity  or  cost.  In  this  regard,  it  is  especially  important  that  analyses  and 
simulations  be  conducted  to  guide  decisions,  with  recognition  of  the  fact  that 
appropriate  complementary  employment  of  systems  can  greatly  enhance  over¬ 
all  Army  combat  effectiveness. 

(Management  T&E.  Methodology)  (Application! 


21.  [GAO,  1980]  MODELS,  DATA,  AND  WAR:  A  CRITIQUE  OF  THE 
FOUNDATION  FOR  DEFENSE  ANALYSES  (GAO  I  PAD-80-2 1) 
(MARCH  1980).  WASHINGTON,  DC:  GENERAL  ACCOUNTING 
OFFICE.  AD-A086  708 


•  We  recommend  the  Secretary  of  Defense  require  the  Chairman,  Joint  Chiefs  of 
Staff,  to  review  current  procedures  for  safeguarding  and  strengthening  the 
empirical-theoretical  foundation  underlying  the  analytic  representation  of 
combat  in  Defense  studies.  This  should  include: 


-  Identifying  the  empirical-theoretical  limitations  of  such  studies  as  are 
employed  in  high-level  Defense  decision-making;  and 

(Investment.  Analysis.  Methodology)  [Targeted  R&D1 


Ascertaining  the  extent  to  which  military  and  civilian  defense  analysts  are 
aware  of  the  empirical  validity  of  their  tools,  and  the  existing  corpus  of 
knowledge  pertaining  to  their  projects. 


(Management,  Analysis.  Information  Sharing! 
Improvement! 


TManaj 


•  And,  as  warranted  by  that  review,  to  prepare  plans  and  recommendations 
which  would  enable  the  Organization  of  the  Joint  Chiefs  of  Staff  to  serve  as  the 
Defense  Establishment's  principal  analytic  adviser  on  matters  pertaining  to  the 
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phenomenology  of  combat  Such  plans  and  recommendations  should  consider 
the  requirements  for: 

-  Developing  adequate  theories  and  empirical  information  about  the 
integrative  structures  of  combat  which  bind  the  Services  together  in  a  web 
of  common  concern  and  interests. 

(Investment  Analysis.  Information  Sharing)  Management 

Improvement! 

-  Making  recommendations  to  the  Secretary  of  Defense  on  priorities  for 
research,  and  on  preferred  theories  and  criteria  to  be  used  in  defense 
analyses;  and 

(Management  R&D.  Methodology)  Management  Improvement! 

~  Establishing  an  institutional  focus  to  provide  the  Defense  analytic 
community  with  (1)  a  senior  adviser,  (2)  a  corporate  memory,  (3)  a 
technical  forum,  and  (4)  a  center  to  disseminate  information  on  the 
techniques  of  combat  analysis. 

(Management  Analysis.  Information  Sharing!  f Coordinate! 

22.  [ASB,  1980]  ARMY  SCIENCE  BOARD  AD  HOC  STUDY  GROUP 
REPORT  ON  HUMAN  ISSUES  (MARCH  1980).  WASHINGTON, 
DC:  ARMY  SCIENCE  BOARD.  [SPONSORED  BY  THE  ASSISTANT 
SECRETARY  OF  THE  ARMY  FOR  RESEARCH,  DEVELOPMENT, 
AND  ACQUISITION]  AD-A091  285 

•  The  Army  should  task  a  full-time  organizational  element  within  the  staff  that 
will  have  authority  to  integrate  and  coordinate  human  issues  policy 
development  research,  analysis,  and  modeling  activities.  The  group  would 
serve  the  purpose  of  elevating  human  issues  considerations  and  combining 
them  with  high  level  policy  determinations.  It  would  become  the  major 
proponent  for  human  issues  model  development  and  applications  within  the 
Army.  It  would  focus  on  Army  needs  rather  than  respond  to  OSD  impetus. 
This  organization  must  receive  high  level  support  and  have  the  authority  and 
proper  mix  of  expertise  to  do  its  job  effectively.  We  suggest  that  it  be  assigned 
at  either  the  Chief  of  Staff  or  Deputy  Chief  of  Staff  level. 

(Management.  Analysis.  Methodology)  f Coordinate! 

•  The  Army  should  develop  holistic  Force  Composition  models  rather  than 
narrow  single  purpose  models.  Human  dimensions  such  as  fatigue  and 
cohesiveness  must  be  quantified  and  included  in  the  Army's  hierarchy  of  Force 
Effectiveness  models.  The  study  group  is  convinced  that  Force  Effectiveness 
models  that  include  human  dimensions  can  be  developed.  Further,  we  believe 
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that  hard-to-quantify  human  characteristics  can  be  quantified  and  included. 
The  study  group  endorses  the  Army's  hierarchical  approach  to  combat 
effectiveness  modeling.  However,  we  urgently  feel  that  the  human  dimension 
must  be  given  full  consideration  in  development  of  each  element  in  the 
hierarchy. 

(Investment.  Analysis.  Methodology)  Haggled  RADI 

•  The  Army  should  establish  a  center  of  excellence  for  human  issues  modeling. 
This  center  should  be  the  focal  point  for  modeling  the  human  component  of 
Force  Effectiveness  and  for  collecting  test  data  necessary  to  support  modeling 
activities.  The  function  of  this  center  would  be  to  relate  current  and  future 
research  findings  [on  such  human  issues  as  fatigue,  leadership,  cohesiveness, 
morale,  and  motivation]  to  maintaining  high  levels  of  readiness,  enhancing 
combat  effectiveness,  predicting  alternative  outcomes,  and  providing  relevant 
human  resource  management  considerations  for  policy  development.  The 
academic  community  and  industry  should  be  solicited  for  inputs  and  assistance 
in  improving  the  state-of-the-art  of  such  modeling  technology. 

(Management.  R&D.  Methodology)  [Coordinate! 

•  The  Army  should  coordinate  its  efforts  [to  develop  integrated  Force 
Composition  models]  with  those  of  the  other  Services  to  eliminate  duplication 
and  to  further  modeling  technology  through  an  active  interchange  of  ideas. 

(Management  Analysis.  Information  Sharing)  [Management  improvement! 


23.  [ASB,  1979]  MILITARY  OPERATIONS  IN  BUILT-UP  AREAS 

(MOBA)  (JANUARY  1979).  WASHINGTON,  DC:  ARMY  SCIENCE 
BOARD.  [SPONSORED  BY  THE  ASSISTANT  SECRETARY  OF  THE 
ARMY  FOR  RESEARCH,  DEVELOPMENT,  AND  ACQUISITION] 
AD-B034  309 


•  MOBA  modeling,  simulation,  and  training  plans  and  activities  should  consider 
the  broad  range  of  MOBA;  namely,  CIC,  operations  involving  the  combination 
of  man-made  towns  and  the  surrounding  natural  tenain,  and  MOUT. 

(Management.  EE&MQ.  Methodology)  [Application! 

•  Field  and  simulation  studies  should  proceed  simultaneously  so  that  each  can 
benefit  from  the  findings  of  the  other. 


(Management,  EE&MQ.  I 


Sharing)  [Technical  Ii 


mi1/  if  *  i 


1] 


•  Continue  the  development  of  URBWAR  as  a  platoon-vs-platoon,  single 
assault  short  time  duration,  shot-by-shot  model,  incorporating  into  it  those 
features  of  other  small-unit-action  simulations  felt  beneficial  and  the  capability 
for  at  least  parametric  treatment  of  other  factors  such  as  CBR.  Some  of  this 
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development  will  have  to  wait  for  the  results  of  field  tests  or  additional  changes 
made  as  new  knowledge  becomes  available. 


llhWl  llir  lilW  ti  ii'ltWiVf  HriVil'i'  'i  i  i'  ■  !  '  I Q I 


Use  URBWAR  to  develop  appropriate  distributions  and  statistics  for  higher 
level  simulations,  models,  and  war  games. 

(Investment  ET&MO.  Methodology)  rTechnical  Improvement! 

Continue  to  develop  MOBACS  as  new  information  becomes  available  from 
field  tests  and  new  subroutines  are  developed  or  adopted  from  other  simulation 
models. 

(Investment  ET&MQ.  Methodology)  rTechnical  Improvement! 

Investigate  the  use  of  intercorrelations  in  the  development  of  the  statistical 
battlefield  in  the  MOBACS  terrain  model  as  a  means  of  increasing  the  realism 
in  representing  street  networks  and  building  complexes. 

(Investment  ET&MQ,  Methodology)  [Targeted  R&Dl 

Develop  a  comprehensive  CIC  model  of  sufficient  resolution  to  investigate 
such  factors  as  squad-level  tactics,  the  details  of  weapon  usage, 
interrelationships  among  the  environment,  tactics,  and  weapons,  individual 
detection  and  firing  capability,  and  psychological  factors.  This  investigation 
should  include  examinations  of  the  MOBACS  Unit  Operations  Level  Game  to 
check  the  feasibility  of  increasing  its  resolution  to  the  level  required  and  of 
DYNTACS  and  perhaps  other  high  resolution  simulations  to  see  if  they  can  be 
adapted  to  MOB  A. 

(Investment,  Analysis,  Methodology)  f  Application! 

Using  the  expertise  gained  by  CAC  in  their  MOBA  study,  modify  (expand) 
SCORES  to  include  MOBA. 

(Investment.  Analysis.  Methodology)  [Targeted  R&Dl 

Investigate  the  appropriateness  of  modifying  JIFFY  or  perhaps  other 
interactive  corps  level  simulations  for  use  as  a  MOBA  MOUT  model  and  as  a 
high  level  training  device. 

(Investment  ET&MO.  Methodology)  [Targeted  R&Dl 

Continue  the  dissemination  of  current  MOBA  doctrine  and  related  material  in 
field  manuals  and  training  documents  and  provide  for  the  rapid  update  of  these 
publications  as  new  and  improved  information  becomes  available. 

(Investment  ET&MO.  Methodology!  [Technical  Improvement! 

Investigate  the  use  of  the  MOBACS  map  exercise  and  Force  Operations  Level 
Game  as  MOBA  training  devices  for  corps  and  division  commanders.  Its 
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interactive  nature  should  make  it  well  suited  for  this  purpose.  Should  this  not 
prove  feasible,  JIFFY  and  other  high  level  models  would  have  to  be 
considered.  If  no  existing  high-level  model  can  be  adapted  to  do  the  job,  a 
new  game  would  have  to  be  developed  from  scratch. 

(Investment.  ET&MO.  Methodology)  [Application! 

•  Appropriate  training  materials  and  inputs  to  the  high  level  MOBA  game  should 
be  prepared  for  use  by  corps  and  division  commanders.  The  following 
subjects  should  be  included:  (1)  the  use  or  urbanized  terrain  as  a  force 
multiplier,  (2)  the  use  of  combined  arms  at  the  corps  level  in  urbanized  terrain, 
(3)  consideration  of  political,  religious,  and  humanitarian  factors,  including  the 
effects  of  collateral  damage,  in  planning  military  operations,  and  (4)  the 
administration  of  non-combatants  in  occupied  areas,  in  particular  where 
fighting  is  taking  place. 

(Investment.  ET&MO.  Methodology)  [Targeted  R&P1 

•  A  MOBA  training  facility  should  be  planned  for  construction  and  modification 
on  a  continuing  basis  as  knowledge  of  MOBA  operations  and  training 
requirements  increases.  An  immediate  capability  should,  however,  be 
provided  that  will  enable  field  exercises  to  be  performed  which  enact  in  real  life 
items  which  the  URBWAR  model  simulates.  This  involves  a  single  attack  on  a 
defended  multi-room  building  starting  either  from  the  street  or  another 
building,  with  platoon-size  forces  on  both  sides.  Results  of  the  field  exercises 
could  then  be  used  to  update  URBWAR  and  URBWAR  study  results  used  to 
organize  and  equip  offensive  and  defensive  forces  in  CIC.  A  logical  next  step, 
as  information  becomes  available,  is  to  provide  for  a  comprehensive  company- 
level  CIC  operation,  eventually  with  combined  arms. 

(Investment.  ET&MO.  Methodology)  [Application! 

24.  [DSB,  1976A]  REPORT  OF  THE  DEFENSE  SCIENCE  BOARD  TASK 
FORCE  ON  TECHNOLOGY  BASE  STRATEGY  (OCTOBER  1976). 
WASHINGTON,  DC:  DEFENSE  SCIENCE  BOARD.  [SPONSORED 
BY  THE  DIRECTOR  OF  DEFENSE  RESEARCH  AND 
ENGINEERING]  AD-A032  372 

•  The  panel  believes  that  the  successes  in  aircrew  training  using  these  new 
approaches  [part  task  trainers  and  full  mission  simulators]  can  be  extended  to  a 
number  of  other  training  areas.  For  example,  the  development  of  Crew, 
Group,  Team,  and  Unit  (CGTU)  training  shows  great  promise.  There  seems 
to  be  a  considerable  number  of  good  ideas  now  in  need  of  further  R&D  to 
develop  them  to  a  useful  state.  In  view  of  the  potential  high  payoff,  the  panel 
recommends  this  as  a  good  candidate  for  increased  funding. 

(Investment,  ET&MO.  Methodology!  [Targeted  R&D! 
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25.  [DSB,  1976B]  REPORT  OF  THE  TASK  FORCE  ON  TRAINING 
TECHNOLOGY  (MARCH  1976).  WASHINGTON,  DC:  DEFENSE 
SCIENCE  BOARD.  [SPONSORED  BY  THE  DIRECTOR  OF 
DEFENSE  RESEARCH  AND  ENGINEERING]  AD-A069  852 

Crew/Group/Teams/Unit  Training 

•  Initiate  systematic  R&D  to  develop  a  taxonomy  of  operational  force  elements 
(crews,  groups,  teams,  and  units),  and  on  methods  for  controlling  training 
variables  in  the  context  of  process-control  models.  This  R&D  should  have 
first  priority  to  establish  a  framework  for  subsequent,  programmatic  R&D  on 
CGTU  training. 

(Investment.  ET&MO.  Methodology)  fTarggMMPI 

•  Incorporate  instructional  technology  into  flight  and  other  simulators  to  improve 
the  effectiveness  of  these  devices  for  training,  and  to  increase  their  utility.  The 
Training  Technology  R&D  elements  of  NPRDC  and  NTEC  (in  the  Navy),  of 
ARI  (in  the  Army),  and  of  AFHRL  (in  the  Air  Force),  should  be  tasked  and 
funded  to  develop  the  instructional  technology  and  the  delivery  systems  to  be 
used  with  these  simulators.  These  laboratories  should  also  participate  in  the 
initial  planning  for  the  simulators,  with  the  responsibilities  for  contributing  the 
training  technology  implementation  and  utilization  plans. 

(Investment.  ET&MO.  Methodology)  rTargeted  R&D1 


A-36 


APPENDIX  B 

RECOMMENDATIONS  CLASSIFIED  BY 
DMSO  GENERAL  OBJECTIVE 

As  Table  B-l  shows,  90  (50  percent)  of  the  179  recommendations  concern  the 
General  Objective  of  Management.  The  Management  recommendations  are  about  evenly 
split  between  the  Functional  Areas  of  Education,  Training,  and  Military  Operations  on  one 
hand  and  Analysis  on  the  other.  Most  of  them  concern  the  Technical  Objective  of 
Methodology. 


Table  B-l.  Management  Objectives 


Recommendations 

Number 

Percent 

Functional  Areas 

Education,  Training,  and  Military  Ops 

31 

34 

Research  and  Development 

13 

14 

Test  and  Evaluation 

11 

12 

Analysis 

33 

37 

Production  and  Logistics 

2 

2 

Total 

90 

Technical  Objectives 

Architectural  Development 

0 

0 

Methodology  Development 

60 

67 

Advanced  Technologies 

3 

3 

Information  Sharing 

27 

30 

Total 

90 

Of  the  90  Management  recommendations: 

•  12  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  5  recommend  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

B*3 


•  20  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  5  recommend  improvements  in  the  processes  for  developing  models  and 
simulations. 

•  33  recommend  general  improvements  for  managing  and  providing  oversight 
for  modeling  and  simulation. 

•  15  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

As  Table  B-2  shows,  14  (8  percent)  of  the  179  recommendations  concern  the 
General  Objective  of  Policy.  Most  of  the  Policy  recommendations  concern  the  Test  and 
Evaluation  Functional  Area  and  the  Technical  Objective  of  Methodology. 


Table  B*2.  Policy  Objectives 


Recommendations 

Number 

Percent 

Functional  Areas 

Education,  Training,  and  Military  Ops 

2 

14 

Research  and  Development 

1 

7 

Test  and  Evaluation 

7 

50 

Analysis 

4 

29 

Production  and  Logistics 

0 

0 

Total 

14 

Technical  Objectives 

Architectural  Development 

0 

0 

Methodology  Development 

10 

71 

Advanced  Technologies 

1 

7 

Information  Sharing 

3 

21 

Total 

14 

Of  the  14  Policy  recommendations: 

•  4  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  None  recommends  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 
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•  1  recommends  technical  improvements  to  modeling  and  simulation  that  require 

application  of  technological  capabilities  already  in  hand. 


•  3  recommend  improvements  in  the  processes  for  developing  models  and 
simulations. 

•  4  recommend  general  improvements  for  managing  and  providing  oversight  for 
modeling  and  simulation. 

•  2  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

As  Table  B-3  shows,  75  (42  percent)  of  the  179  recommendations  concern  the 
General  Objective  of  Investment.  These  recommendations  are  about  evenly  split  between 
the  Functional  Areas  of  Education,  Training,  and  Military  Operations  on  one  hand  and 
Analysis  on  the  other.  Most  of  them  concern  the  Technical  Objective  of  Methodology. 


Table  B-3.  Investment  Objectives 


Recommendations 

Number 

Percent 

Functional  Areas 

Education.  Training,  and  Military  Op 

30 

40 

Research  and  Development 

9 

12 

Test  and  Evaluation 

6 

8 

Analysis 

28 

37 

Production  and  Logistics 

2 

3 

Total 

75 

Technical  Objectives 

Architectural  Development 

1 

1 

Methodology  Development 

52 

69 

Advanced  Technologies 

14 

19 

Information  Sharing 

8 

11 

Total 

75 

Of  the  75  Investment  recommendations: 

•  13  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  29  recommend  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 
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18  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

4  recommend  improvements  in  the  processes  for  developing  models  and 
simulations. 

8  recommend  general  improvements  for  managing  and  providing  oversight  for 
modeling  and  simulation. 

3  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 


APPENDIX  C 

RECOMMENDATIONS  CLASSIFIED  BY 
DMSO  FUNCTIONAL  AREA 


As  Table  C-l  shows,  63  (35  percent)  of  the  179  recommendations  concern 
modeling  and  simulation  applications  in  Education,  Training,  and  Military  Operations. 
These  recommendations  are  about  evenly  split  between  the  General  Objectives  of 
Management  and  Investment.  Most  of  them  concern  the  Technical  Objective  of 
Methodology. 


Table  C-i.  Education,  Training,  and  Military  Operations  Functional  Area 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

31 

49 

Policy 

2 

3 

Investment 

30 

48 

Total 

63 

Technical  Objectives 

Architectural  Development 

1 

2 

Methodology  Development 

51 

81 

Advanced  Technologies 

4 

6 

Information  Sharing 

7 

11 

Total 

63 

Of  the  63  Education,  Training,  and  Military  Operations  recommendations: 

•  13  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  14  recommend  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

•  17  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 
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•  None  recommends  improvements  in  the  processes  for  developing  models  and 
simulations. 

•  14  recommend  general  improvements  for  managing  and  providing  oversight 
for  modeling  and  simulation. 

•  5  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

As  Table  C-2  shows,  23  (13  percent)  of  the  179  recommendations  concern 
modeling  and  simulation  applications  in  Research  and  Development.  Most  of  these 
recommendations  concern  the  General  Objective  of  Management  and  the  Technical 
Objective  of  Methodology. 


Table  C-2.  Research  and  Development  Functional  Area 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

13 

57 

Policy 

1 

4 

Investment 

9 

39 

Total 

23 

Technical  Objectives 

Architectural  Development 

0 

0 

Methodology  Development 

12 

52 

Advanced  Technologies 

9 

39 

Information  Sharing 

2 

9 

Total 

23 

Of  the  23  Research  and  Development  recommendations: 

•  1  recommends  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  8  recommend  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

•  4  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  3  recommend  improvements  in  the  processes  for  developing  models  and 
simulations. 


•  4  recommend  general  improvements  for  managing  and  providing  oversight  for 
modeling  and  simulation. 

•  3  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

As  Table  C-3  shows,  24  (13  percent)  of  the  179  recommendations  concern 
modeling  and  simulation  applications  in  Test  and  Evaluation.  Most  of  these 
recommendations  concern  the  General  Objective  of  Management  and  the  Technical 
Objective  of  Methodology. 


Table  C-3.  Test  and  Evaluation  Functional  Area 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

11 

46  ' 

Policy 

7 

29 

Investment 

6 

25 

Total 

24 

Technical  Objectives 

Architectural  Development 

0 

0 

Methodology  Development 

18 

75 

Advanced  Technologies 

1 

4 

Information  Sharing 

5 

21 

Total 

24 

Of  the  24  Test  and  Evaluation  recommendation s : 

•  8  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  None  recommends  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

•  2  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  9  recommend  improvements  in  the  processes  for  developing  models  and 
simulations. 
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•  3  recommend  general  improvements  for  managing  and  providing  oversight  for 
modeling  and  simulation. 

•  2  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

As  Table  C-4  shows,  65  (36  percent)  of  the  179  recommendations  concern 
modeling  and  simulation  applications  in  Analysis.  These  recommendations  are  evenly  split 
between  the  General  Objectives  of  Management  and  Investment  and  emphasize  the 
Technical  Objective  of  Methodology. 


Table  04.  Analysis  Functional  Area 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

33 

51 

Policy 

4 

6 

Investment 

28 

43 

Total 

65 

Technical  Objectives 

Architectural  Development 

0 

0 

Methodology  Development 

39 

60 

Advanced  Technologies 

4 

6 

Information  Sharing 

22 

34 

Total 

65 

Of  the  65  Analysis  recommendations: 

•  6  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  12  recommend  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

•  16  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  None  recommends  improvements  in  the  processes  for  developing  models  and 
simulations. 

•  24  recommend  general  improvements  for  managing  and  providing  oversight 
for  modeling  and  simulation. 


•  7  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 

centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

As  Table  C-5  shows,  only  four  (2  percent)  of  the  179  recommendations  concern 
modeling  and  simulation  applications  in  Production  and  Logistics.  Two  address  the 
General  Objective  of  Management  and  two  concern  Investment  All  four  address  the 
Technical  Objective  of  Methodology. 


Table  C-5.  Production  and  Logistlca  Functional  Area 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

2 

50 

Policy 

0 

0 

Investment 

2 

50 

Total 

4 

Technical  Objectives 

Architectural  Development 

0 

0 

Methodology  Development 

4 

100 

Advanced  Technologies 

0 

0 

Information  Sharing 

0 

0 

Total 

4 

APPENDIX  D 

RECOMMENDATIONS  CLASSIFIED  BY 
DMSO  TECHNICAL  OBJECTIVE 


Only  one  recommendation  concerns  Architectural  Development  It  addresses  the 
General  Objective  of  Investment  and  the  Functional  Area  of  Education,  Training,  and 
Military  Operations.  It  recommends  that  joint  simulations  be  made  interoperable  through 
standardization  so  that  they  can  be  networked  and  operated  from  physically  dispersed  duty 
stations. 


Table  D-i.  Architectural  Development  Technical  Objective 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

0 

0 

Policy 

0 

0 

Investment 

1 

100 

Total 

1 

Functional  Areas 

Education,  Training,  and  Military  Ops 

1 

100 

Research  and  Development 

0 

0 

Test  and  Evaluation 

0 

0 

Analysis 

0 

0 

Production  and  Logistics 

0 

0 

Total 

1 

As  Table  D-2  shows,  123  (69  percent)  of  the  179  recommendations  concern 
Methodology.  However,  Methodology  was  a  catch-all  Technical  Objective  for  many  of  the 
recommendations.  The  closeness  with  which  these  recommendations  are  linked  to 
methodological  considerations  in  modeling  and  simulation  varies  widely.  These 
recommendations  are  evenly  split  between  the  General  Objectives  of  Management  and 
Investment  and  between  the  Functional  Areas  of  Education,  Training,  and  Military 
Operations  and  Analysis. 
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Table  D*2.  Methodology  Technical  Objective 

I  Recommendations 


Number 

Percent 

General  Objectives 

Management 

60 

49 

Policy 

il 

9 

Investment 

52 

42 

Total 

123 

Functional  Areas 

Education,  Training,  and  Military  Ops 

51 

41 

Research  and  Development 

12 

10 

Test  and  Evaluation 

18 

15 

Analysis 

38 

31 

Production  and  Logistics 

4 

3 

Total 

123 

Of  the  123  Methodology  recommendations:  / 

•  28  recommend  extending  modeling  and  simulation  technology  to  tt^geted  areas 
of  application. 

•  25  recommend  research  and  development  to  achieve  specifically  targeted 

improvements  in  modeling  and  simulation.  /' 

•  27  recommend  technical  improvements  to  modeling^and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  8  recommend  improvements  in  the  processes  for  developing  models  and 

simulations.  x  / 

r 

•  26  recommend  general  improvements  for  managing  and  providing  oversight 
for  modeling  and  simulation. 

•  10  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

Advanced  Technology  could  reasonably  be  combined  with  Methodology  as  a 
Technical  Objective,  but  it  is  kept  separate  for  this  analysis.  As  Table  D-3  shows,  18 
(10  percent)  of  the  179  recommendations  concern  Advanced  Technology.  These 
recommendations  emphasize  the  General  Objective  of  Investment  and  the  Functional  Area 
of  Research  and  Development. 
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Table  D-3.  Advanced  Technology  Technical  Objective 


Recommendations 

Number 

Percent 

General  Objectives 

Management 

3 

17 

Policy 

1 

6 

Investment 

14 

78 

Total 

18 

Functional  Areas 

Education,  Training,  and  Military  Ops 

4 

22 

Research  and  Development 

9 

50 

Test  and  Evaluation 

1 

6 

Analysis 

4 

22 

Production  and  Logistics 

0 

Total 

18 

Of  the  18  Advanced  Technology  recommendations: 

•  None  recommends  extending  modeling  and  simulation  technology  to  targeted 
areas  of  application. 

•  1 1  recommend  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

•  3  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  3  recommend  improvements  in  the  processes  for  developing  models  and 
simulations. 

•  None  recommends  general  improvements  for  managing  and  providing 
oversight  for  modeling  and  simulation. 

•  1  recommends  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 

Information  Sharing  could  reasonably  be  combined  with  Methodology  as  a 
Technical  Objective,  but  it  is  kept  separate  for  this  analysis.  As  Table  D-4  shows,  37 
(21  percent)  of  the  179  recommendations  concern  Information  Sharing.  These 
recommendations  emphasize  the  General  Objective  of  Management  and  and  the  Functional 
Area  of  Analysis. 
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Table  D-4.  Information  Sharing  Technical  Objective 


Recommendations  I 

Number 

Percent 

General  Objectives 

Management 

27 

73 

Policy 

2 

5 

Investment 

8 

22 

Total 

37 

Functional  Areas 

Education,  Training,  and  Military  Ops 

7 

19 

Research  and  Development 

2 

5  i 

Test  and  Evaluation 

5 

14 

Analysis 

23 

62 

Production  and  Logistics 

0 

0 

Total 

37 

Of  the  37  Information  Sharing  recommendations: 

•  2  recommend  extending  modeling  and  simulation  technology  to  targeted  areas 
of  application. 

•  1  recommends  research  and  development  to  achieve  specifically  targeted 
improvements  in  modeling  and  simulation. 

•  8  recommend  technical  improvements  to  modeling  and  simulation  that  require 
application  of  technological  capabilities  already  in  hand. 

•  1  recommends  improvements  in  the  processes  for  developing  models  and 
simulations. 

•  17  recommend  general  improvements  for  managing  and  providing  oversight 
for  modeling  and  simulation. 

•  8  recommend  establishment  of  centers  for  modeling  and  simulation  or  of 
centralized  authority  for  managing  modeling  and  simulation  research, 
development,  and  practice. 


APPENDIX  E 

RECOMMENDATIONS  CLASSIFIED  BY 
RECOMMENDED  ACTION 


The  recommendations  may  be  summarized  under  each  of  these  six  areas  of 
recommended  action  (Table  E-l). 


Table  E-l.  Tally  of  Recommended  Actions 


Number 

Percent 

Actions 

Application  Area 

29 

16 

Targeted  Research  and  Development 

34 

19 

Technical  Improvement 

39 

22 

Modeling  and  Simulation  Development 

12 

7 

Management  Improvement 

45 

25 

Coordination 

20 

11 

Total 

179 

1 .  APPLICATION  AREA 

Many  study  panels,  addressing  various  issues  of  military  policy  and  practice,  have 
recommended  modeling  and  simulation  as  means  for  resolving  these  issues.  As  Table  E-l 
shows,  29  (16  percent)  of  the  179  recommendations  concern  either  applications  of 
modeling  and  simulation  to  new  areas  of  activity  or  expansions  of  the  use  of  modeling  and 
simulation  in  already  existing  areas  of  application.  These  applications  include  at  least  the 
following  16  areas: 

•  Military  operations  in  built  up  areas  (MOBA)  [ASB,  1979]; 

•  Determining  tradeoffs  among  performance  and  the  reliability/availability/ 
maintainability  of  electronic  systems  and  tests  of  their  interoperability  [ASB, 
1981]; 

•  Evaluating  the  military  worth  and  costs  of  new  weapons  concepts  ("simulate 
before  decide")  [DSB,  1982b]; 

•  Selected  operation  training  programs  [JDL,  1983]; 
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•  Training  to  develop  leaders  and  units  for  airland  battle  [ASB,  1983]; 

•  Determining  conventional  munitions  acquisition  objectives  [DSB,  1985]; 

•  Officer/NCO  institutional  training  [ASB,  1985a]; 

•  Personal  rapid  experimentation  in  the  operational  art/tactics  of  airland  battle 
doctrine  for  General  Officers  [ASB,  1985a]; 

•  Long  term  mission  preparedness  for  Third  World  urban  conflicts  [DSB,  1986]; 

•  Army,  Marine  Corps,  Navy  joint  campaigns  involving  ECM  and  ECCM 
[ASB,  1988]; 

•  Multi-corps  to  Army  group  simulation  for  joint  operations  [ASB,  1988]; 

•  Echelon  above  corps  for  senior  commander  training  [ASB,  1988]; 

•  On-station  training  for  CINCs  [DSB,  1988]; 

•  Establishing  a  major  joint  training  initiative  coordinated  by  the  Chief,  Joint 
Staff  [DSB,  1988]; 

•  Systematic  use  of  excursion  analyses  (to  include  man-in-the-loop  simulation) 
in  all  phases  of  acquisition,  test,  and  evaluation  [DSB,  1989]; 

•  Development  of  major  materiel  systems  in  which  simulation  using  electronic 
prototypes  complements  testing  the  actual  articles  in  the  field  [ASB,  1991]. 

2 .  TARGETED  RESEARCH  AND  DEVELOPMENT 

Many  study  panels  have  noted  requirements  for  increased  capabilities  in  the 
technology  of  modeling  and  simulation.  As  Table  E-l  (above)  shows,  34  (19  percent)  of 
the  179  recommendations  concern  research  and  development  of  new  or  expanded 
capabilities.  About  two-thirds  of  these  recommendations  concern  specific  capabilities  that 
should  be  developed,  and  about  one-third  concern  the  management  of  programs  to 
accomplish  these  ends.  These  recommendations  include  development  of  the  following: 

•  A  taxonomy  of  operational  force  elements  (crews,  groups,  teams,  and  units) 
and  of  methods  for  controlling  their  processes  [DSB,  1976b]; 

•  Instructional  technology  incorporated  into  simulators  to  increase  their 
effectiveness  and  utility  [DSB,  1976b]; 

•  Part  task  trainers  and  full  mission  simulators  for  crew,  group,  team,  and  unit 
training  [DSB,  1976a]; 

•  A  platoon  versus  platoon,  single  assault,  short  time  duration,  shot  by  shot 
model  for  military  operations  in  built-up/urban  areas  [ASB,  1979]; 


•  Interactive,  corps  level  simulation  for  use  in  high  level  training  for  military 
operations  in  built-up/urban  areas  [ASB,  1979]; 

•  Human  dimensions  such  as  fatigue  and  cohesiveness  incorporated  into  force 
composition  and  force  effectiveness  models  [ASB,  1980]; 

•  Identification  of  the  empirical-theoretical  limitations  of  models  and  data 
employed  in  high-level  Defense  decision  making  [GAO,  1980]; 

•  Increased  use  of  intelligent  adversaries,  realistic  conditions,  and  advanced 
technology  (e.g.,  arcade-like  devices  and  simulations)  in  training  simulations 
[DSB,  1982a]; 

•  Improved  visual  and  imaging  capabilities  applied  to  training  simulators  [JDL, 
1983]; 

•  Combat  service  support  force  structure  planning  models  [ASB,  1985b]; 

•  New  technologies  incorporated  in  training  simulations  to  reduce  controller 
requirements  and  increase  their  portability  [ASB,  1985a]; 

•  Accelerated  SIMNET  development  and  its  connection  with  larger  battle 
simulations  [ASB,  1985a]; 

•  An  echelon  above  corps  (EAC)  joint  campaign  model  [ASB,  1988]; 

•  An  "intelligent  analyzer"  expert  system  and  "analytic  analog"  campaign  models 
[ASB,  1988]; 

•  Distributed  wargames  and  models  to  allow  remote  participation  [ASB,  1988]; 

•  An  exploratory  program  to  demonstrate  programs  to  satisfy  long-term  joint 
training  simulation  requirements  [GAO,  1988]; 

•  Networked  simulation  applicable  to  both  active  and  reserve  components  for 
close  combat  (heavy)  training  [ASB,  1989]; 

•  Continued  funding  for  modeling  and  simulation  by  DDR&E  at  both  the 
fundamental  and  application  levels  [DSB,  1989]; 

•  Rapid  adaptation  to  new  threats  and  unanticipated  contingencies  and  treatment 
of  reserve  components  as  mainline  elements  in  simulation  [FM&P,  1990]; 

•  Programs  that  invest  15  percent  of  available  modeling  and  simulation  funds  in 
basic  research  [FM&P,  1990]; 

•  Programs  that  invest  60  percent  of  available  modeling  and  simulation  funds  to 
develop  a  prototype  representation  of  an  operational  theater  that  draws  on 
empirical  data  derived  from  physical  and  human  systems  [FM&P,  1990]; 

•  Programs  funded  at  the  Department  of  Army  level  to  extend  existing  SIMNET 
capabilities  [ASB,  1991]. 
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3 .  TECHNICAL  IMPROVEMENT 


Many  study  panels  have  noted  requirements  to  extend  application  of  existing 
technical  capabilities  in  the  conduct  of  modeling  and  simulation.  As  Table  E-l  shows, 
39  (22  percent)  of  the  179  recommendations  concern  improvements  that  require  better  use 
of  existing  technical  capabilities.  Among  the  technical  improvements  that  have  been 
recommended  are  the  following: 

•  Field  and  simulation  studies  should  proceed  simultaneously  so  that  each 
benefits  from  the  other  and  disseminate  their  results  rapidly  in  field  manuals 
and  training  documents  [ASB,  1979]; 

•  Develop  and  provide  a  common  library  of  software  verification  and  validation 
tools  [ASB,  1981]; 

•  Assess  the  impact  of  the  manpower  pool  on  new  weapons  systems  and  identify 
where  training  may  increase  the  performance  of  new  recruits  in  meeting  system 
needs  [DSB,  1982a]; 

•  Discipline  the  requirement  for  RAM/ILS  during  development  of  new  systems 
and  the  collection  of  field  logistics  data  [ASB,  1985b]; 

•  Align  training  simulation  models  more  closely  with  models  used  for  analysis 
[ASB,  1985a]; 

•  Identify  devices,  simulations,  methods,  and  practices  that  require  validation, 
then  provide  it  [ASB,  1985a]; 

•  Institutionalize  from  the  beginning  intelligence  community  participation  in 
Third  World  operations  planning,  gaming,  simulations,  and  exercises  [DSB, 
1986]; 

•  Correct  noted  limitations  in  validation  of  ADAGE,  CARMONETTE,  and 
COMO  ffl  [GAO,  1987]; 

•  Include  dynamics  of  the  onset  of  battle  conditions  in  joint  scenarios  [ASB, 
1988]; 

•  Replace  assessment  routines  in  training  models  with  those  of  analysis  models 
[ASB,  1988]; 

•  Improve  the  Army's  consideration  of  combined  forces  in  its  analyses  of  U.S. 
defense  issues  and  combined  issues  [ASB,  1988]; 

•  Improve  consideration  of  CINCs'  needs  in  Army  analyses  [ASB,  1988]; 

•  Review  the  adequacy  of  joint  scenarios,  suggest  more  appropriate  cmes  and  a 
process  to  ensure  their  consideration  [ASB,  1988]; 

•  Require  parametric  analysis  in  study  directives  [ASB,  1988]; 


+ 
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•  Improve  the  assessment  routines  (e.g.,  in  MTM,  JTLS,  and  JESS)  used  in 
warfighting  training  for  commanders  and  staffs  [ASB,  1988]; 

•  Model  development  should  be  evolutionary  and  iterative  in  conjunction  with 
studies--not  "bold  leaps"  [ASB,  1988]; 

•  Foster  the  Warrior  Preparation  Center  concept  in  unified  commands  and  senior 
Service  colleges  [ASB,  1988]; 

•  Use  model  subset  libraries  and  relational  databases  to  simplify  and  speed  up 
the  development  of  computer  models  [ASB,  1988]; 

•  Either  purchase  or  authorize  rental  of  contemporary  high-speed  data  processing 
equipment  [ASB,  1988]; 

•  Adapt  commercial  graphics  improvements  to  aid  data  interpretation  [ASB, 
1988]; 

•  Address  the  apparent  lack  of  transportability  in  diverse  artificial  intelligence 
hardware  and  software  [ASB,  1988]; 

•  Internet,  with  standardized  communication  protocols,  existing  games  and 
simulations  for  training  so  that  they  are  accessible  from  duty  stations  [DSB, 

1988] ; 

•  Increase  use  of  command  and  staff  simulations  to  allow  higher  echelons  to 
train  up  prior  to  field  exercises  [ASB,  1989]; 

•  Further  tailor  combat  training  to  the  unique  requirements  of  the  reserve 
components  through  increased  use  of  simulation  and  networking  integrated 
with  those  used  by  the  active  components  [ASB,  1989]; 

•  Establish  Held  and  simulator  test-beds  with  supporting  simulation  models  for 
new  training  strategies,  devices,  and  techniques  comparable  to  materiel  testing 
facilities  [ASB,  1989]; 

•  Ensure  the  appropriate  use  of  engineering  simulations  as  test-beds  to  identify 
and  reduce  excessive  training  burdens  of  new  and  modified  equipment  [ASB, 

1989] ; 

•  Exploit  technology  opportunities  in  distributed  computing  and  networking  to 
simulate  combined  arms  engagements  coordinated  with  man-in-the-loop 
simulations  and  to  evaluate  their  results  against  live  fire  exercises  [DSB,  1989]; 

•  Invest  15  percent  of  modeling  and  simulation  funds  in  enabling  technologies 
(e.g.,  more  powerful,  cheaper  processors,  graphic  displays,  improved 
communications)  [FM&P,  1990]; 

•  Adopt  a  combined  arms  simulation  strategy  from  the  outset  [ASB,  1991]. 
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4 .  MODELING  AND  SIMULATION  DEVELOPMENT 


As  Tabic  E-l  shows,  12  (7  percent)  of  the  179  recommendations  concern 
improvements  in  the  design  and  development  of  models  and  simulations.  Almost  all  of 
these  recommendations  concern  the  development  of  models  and  simulations  used  in  the 
acquisition,  test,  and  evaluation  of  other  systems.  These  improvements  include  the 
following: 

•  Testing  and  user  activities  should  quantify  and  document  test  requirements  for 
environment  simulators  used  in  the  test  and  evaluation  of  electronic  systems 
[ASB,  1981]; 

•  Requirements  for  the  preparation  and  approval  of  test  and  evaluation  master 
plans  (TEMPs)  should  be  enforced  [GAO,  1983]; 

•  TEMPs  should  state  existing  abilities  and  inabilities  to  assess  a  system's 
operational  suitability  and  effectiveness  [GAO,  1983]; 

•  DoD-wide  guidance  should  be  developed  on  producing,  validating, 
documenting,  managing,  maintaining,  using,  and  reporting  simulations  of 
system  effectiveness  [GAO,  1987]; 

*-  A  model  deficiency  analysis  and  prioritization  should  be  conducted  along  with 
a  technology  forecast  to  design  a  research  program  in  campaign  modeling  and 
analysis  [ASB,  1988]; 

•  Technology  for  developing  models  and  simulations  should  be  adopted  as  much 
as  possible  from  elsewhere  and  developed  only  for  Army-unique  needs  [ASB, 
1991]. 

5 .  MANAGEMENT  IMPROVEMENT 

As  Table  E-l  shows,  45  (25  percent)  of  the  179  recommendations  concern 
improvements  in  the  way  modeling  and  simulation  activities  are  conducted,  controlled,  and 
used.  These  recommendations  included  the  following: 

•  The  Army  should  coordinate  its  efforts  to  develop  integrated  Force 
Composition  models  with  those  of  the  other  Services  [ASB,  1980]; 

•  Review  the  extent  to  which  military  and  civilian  defense  analysts  are  aware  of 
the  empirical  validity  of  their  models  [GAO,  1980]; 

•  Develop  plans  and  recommendations  that  allow  the  Organization  of  the  Joint 
Chiefs  of  Staff  to  serve  as  the  Defense  Establishment's  principal  analytic 
adviser  on  the  phenomenology  of  combat  [GAO,  1980]; 


•  Review  the  Defense  intelligence  community  to  identify  and  solve  problems 
preventing  its  adequate  support  of  threat  simulator  development  [GAO,  1983]; 

•  Organizations  responsible  for  validating  training  devices,  simulations, 
methods,  and  practices  should  be  adequately  resourced  and  independently 
reviewed  [ASB,  1985a]; 

•  The  Army  should  re-consider  limits  on  data  exchange  with  NATO  allies 
[ASB,  1988]; 

•  Provide  peer  review  of  Army  models  and  model  development  [ASB,  1988]; 

•  Use  wargame  training  models  to  identify  analysis  issues,  but  not  to  perform 
analysis; 

•  Deputy  Undersecretary  of  the  Army  for  Operations  Research  should  take  a 
leadership  position  with  J-8  regarding  Army  analysis  [ASB,  1988]; 

•  Vice  Chief  of  Staff  of  the  Army  and  Vice  Chief  of  Naval  Operations  should 
form  a  joint  analysis  steering  committee  for  campaign  models  and  analysis 
[ASB,  1988]; 

•  The  Joint  Analysis  Steering  Group  should  take  the  lead  in  fostering  the 
development  of  joint  inputs  for  submission  to  J-8  [ASB,  1988]; 

•  Quality  Army  personnel  should  be  made  available  for  assignment  to  J-8  and  the 
staffs  of  the  Unified  Commands  [ASB,  1988]; 

•  Deputy  Undersecretary  of  the  Army  should  ensure  that  the  Army  analysis 
community  participates  with  J-8  in  supporting  the  CINCs'  needs  [ASB,  1988]; 

•  Establish  an  Army  analysis  improvement  program  using  senior  analysts  as 
trainers  [ASB,  1988]; 

•  Select  5-10  studies  annually  for  close  personal  participation  and  technical 
review  [ASB,  1988]; 

•  Maintain  a  small  number  of  joint  analysis  teams  doing  continual  analysis  of 
airland  campaign  issues  [ASB,  1988]; 

•  TRADOC  should  establish  a  pro-active  cell  of  analysts  and  operations/doctrine 
experts  to  perform  continuous  analysis  of  joint/combined  tactics,  operations, 
and  doctrine  [ASB,  1988]; 

•  Models  used  in  Army  joint  studies  should  be  technically  understood  by 
multiple  agencies  with  a  direct  audit  trail  to  objective,  measurable  data  [ASB, 
1988]; 

•  AMMO's  mission  should  include  execution  of  joint/combined  modeling 
developments  and  coordinating  this  work  with  outside  agencies  [ASB,  1988]; 
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•  DARPA  should  host  a  semi-annual  joint  modeling  research  seminar  [ASB, 

1988] ; 

•  The  Services  should  strengthen  internal  controls  over  simulator  acquisition  by 
segregating  responsibilities  for  development,  testing,  and  acceptance  of  threat 
simulators  [GAO,  1988]; 

•  Establish  the  requirements  for  long-term  joint  simulation  and  document  them  in 
JCS  directives  [DSB,  1988]; 

•  The  Assistant  Secretary  of  Defense  for  Force  Management  and  Personnel 
should  participate  in  establishing  policies  and  procedures  for  wargaming  and 
similar  activities  [IG,  1989]; 

•  Discontinue  operations  at  the  Joint  Warfare  Center,  discontinue  their  funding, 
and  transfer  management  oversight  responsibilities  to  the  Joint  Staff  [IG, 

1989] ; 

•  The  Naval  War  College  should  request  a  waiver  from  the  Joint  Staff  to  allow 
the  development,  testing,  and  analysis  of  operational  war  plans  and 
contingency  plans.  If  denied  the  War  College  should  discontinue  its  support  of 
testing  and  analysis  of  plans  [IG,  1989]; 

•  The  DCNO,  Naval  Warfare,  should  stop  constructing  Sensitive 
Compart mented  Information  Facilities  at  Navy  schools  when  wargaming 
activities  are  limited  to  education  and  training  [IG,  1989]; 

•  Further  development  of  the  Command  Readiness  Exercise  System  by  the  Air 
Force  should  be  coordinated  with  other  Service  schools  [IG,  1989]; 

•  Requirements  for  the  Command  Readiness  Exercise  System  should  be  made 
consistent  with  Joint  Chiefs  of  Staff  policy  on  distribution  of  war  plans  and 
contingency  plans  [IG,  1989]; 

•  The  Warrior  Preparation  Center  should  prepare  an  updated  management  plan 
specifying  its  objectives,  operating  procedures,  and  facilities  requirements  [IG, 
1989]; 

•  Discontinue  the  $40  million  in  funding  required  by  the  Warrior  Preparation 
Center  to  expand  its  facilities,  procure  computer  systems,  and  develop 
computer  models  [IG,  1989]; 

•  Ensure  that  the  Warrior  Preparation  Center  includes  U.S.  Army,  Europe,  U.S. 
Naval  Forces,  Europe,  and  the  U.S.  Air  Force  Europe  in  its  wargaming 
exercises  [IG,  1989]; 

•  Ensure  that  the  U.S.  Government  is  reimbursed  by  NATO  nations  that 
participate  in  wargaming  exercises  held  at  the  Warrior  Preparation  Center  [IG, 
1989]; 
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Re-examine  the  role  and  implementation  of  Manprint  during  systems 
acquisition/to  ensure  the  appropriate  use  of  engineering  simulations  to  identify 
and  reduce  potentially  excessive  training  burdens  of  new  and  modified 
equipment  [ASB,  1989]; 

Independent  panels  of  expens  should  assess  specific  applications  of  modeling 
and  simulation  results  on  which  acquisition  decisions  are  based  [DSB,  1989]; 

Modify  DoDI  5000.2  to  require  that  DAB  documentation  address  the 
applicability  of  models  and  simulation  [DSB,  1989]; 

Approve  and  implement  the  EXCIMS/DMSO  management  scheme.  DARPA 
should  remain  the  lead  OSD  actor  in  R&D  but  pass  overall  management  to  the 
DMSO  [DSB,  1989]; 

Establish  policies  to  ensure  full  integration  of  military  science  with  the 
acquisition  of  modeling  and  simulation  including  the  education  of  military 
service  personnel  in  modeling  and  simulation  developments  and  applications 
[FM&P,  1990]; 

The  Army  should  review  the  current  structure  of  the  Electronic  Battlefield 
program  [ASB,  1991]; 

The  Army  should  forego  OPTEMPO  as  well  as  other  funds  to  finance  the  new 
modes  of  training  [ASB,  1991]. 

6.  COORDINATION 

As  Table  E-l  shows,  20  (11  percent)  of  the  179  recommendations  concern  the 
coordination  of  modeling  and  simulation  activities  or  resources  under  single  management  or 
in  a  single  location.  These  coordination  recommendations  are  a  subset  of  the  management 
improvements  discussed  above.  They  include  the  following: 

•  The  Army  should  task  a  full-time  organizational  element  to  integrate  and 
coordinate  human  issues  policy  development,  research,  analysis,  and  modeling 
activities  [ASB,  1980]; 

•  The  Army  should  establish  a  center  of  excellence  for  human  issues  modeling 
[ASB,  1980]; 

•  The  Organization  of  the  Joint  Chiefs  of  Staff  should  establish  an  institutional 
focus  to  provide  the  Defense  analytic  community  with  (1)  a  senior  advisor,  (2) 
a  corporate  memory,  (3)  a  technical  forum,  and  (4)  an  information 
dissemination  center  [GAO,  1980]; 

•  Create  a  private  sector  organization  as  a  core  work  force  for  the  development  of 
long-term,  professional  quality  in  simulation  techniques  [DDR&E,  1982]; 
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•  Establish  a  joint-Service  threat  simulator  and  aerial  target  improvement 
program  to  identify,  time  phase,  and  prioritize  DoD-wide  test  resource 
deficiencies  [GAO,  1983]; 

•  Support  the  Hermann  model  of  a  tri-Service  training  and  simulation  R&D 
center  with  close  university  relations  [JDL,  1983]; 

•  The  Vice  Chief,  Army,  and  the  Vice  Chief,  Air  Force,  should  form  a  joint 
analysis  steering  group  at  the  headquarters  level  [ASB,  1988]; 

•  This  joint  analysis  steering  group  should  resolve  disparities  among  existing 
joint  models  and  consider  the  development  of  joint  scenarios  and  data  bases  for 
analyses  [ASB,  1988]; 

•  Chief  of  Staff,  Army,  and  Chief  of  Naval  Operations  agree  to  examine  specific 
areas  of  mutual  cooperation  in  planning  [ASB,  1988]; 

•  Deputy  Undersecretary  of  the  Army,  Operations  Research,  should  take  the  lead 
in  ensuring  Army's  continuation  of  a  strong  liaison  with  J-8  [ASB,  1988]; 

•  Deputy  Undersecretary  of  the  Army,  Operations  Research,  should  form  a 
Department  of  the  Army  level  group  comprised  of  senior  experienced 
professionals  to  structure  and  implement  procedures  for  peer  review  of  Army 
campaign  models  [ASB,  1988]; 

•  The  Secretary  of  Defense  should  ensure  that  appropriate  DoD  elements  execute 
responsibility  and  authority  for  centrally  managing  simulator  programs  [GAO, 
1988]; 

•  The  Secretary  of  Defense  should  assign  to  an  appropriate  DoD  element  the 
responsibility  for  monitoring  the  quality  of  simulators  acquired  and 
participating  in  the  acquisition  process  [GAO,  1988]; 

•  Create  a  shared  library  of  automated,  validated  data  descriptions  for  simulation 
use  that  are  available  DoD-wide  [DSB,  1988]; 

•  The  Director,  Joint  Staff,  should  designate  an  office  of  primary  responsibility 
within  the  Joint  Staff  to  establish  policies  and  procedures  on  wargaming, 
simulations,  modeling,  and  exercises  and  clarify  the  responsibilities  and 
relationships  of  DoD  participants  in  joint  wargaming  [IG,  1989]; 

•  The  Undersecretary  of  Defense,  Acquisition,  should  ensure  refinement, 
maintenance,  and  availability  of  models,  weapon  and  threat  data  descriptions, 
and  simulation  elements  having  DoD-wide  utility  [DSB,  1989]; 

•  The  Electronic  Battlefield  program  should  be  funded  at  the  Department  of 
Army  level  with  a  single  manager  and  a  direct  mandate  from  the  Secretary  of 
the  Army  [ASB,  1991]; 
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•  A  single  manager  should  be  put  in  charge  of  the  Electronic  Battlefield  [ASB, 
1991]; 

•  The  Army  should  develop  a  process  and  discipline  to  ensure  quality  of  data 
across  the  total  spectrum  of  model  use-this  should  be  a  prime  task  of  the 
manager  of  the  Electronic  Battlefield  [ASB,  1991]. 
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